Smartly calculate real running time and pending time
## What does this MR do?
Try to smartly calculate the running time and pending time for pipelines, instead of just use wall clock time from start to end. The algorithm is based on:
> Suppose we have A, B, and C jobs:
> * A: from 1 to 3
> * B: from 2 to 4
> * C: from 6 to 7
> The processing time should be accumulated from 1 to 4, and 6 to 7, totally 4, excluding retires, and calculate on `%w[success failed running canceled]` jobs (if a job is not finished yet, assume it's `Time.now`)
## Are there points in the code the reviewer needs to double check?
I would actually like to test `Gitlab::Ci::PipelineDuration#process_segments`, but it's a private method right now and it's not very convenient to test it. Is there a way to test it without changing the original code too much? Note that I would like to avoid saving merged segments because it's not used and should be garbage collected.
## Screenshots:
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
- [x] Added for this feature/bug
## What are the relevant issue numbers?
Closes#18260, #19804
See merge request !6084
* master: (414 commits)
Remove suggested colors hover underline
Fix markdown anchor icon interaction
Fix expiration date picker after update
Refactored code to rely less on IDs that could change
Move CHANGELOG entry for !5858 from 8.11 to 8.12
Hides merge request section in edit project when disabled
Fix a typo
Change minimum Unicorns required to two
Update memory requirements
Added `.term-bold` declaration.
Change the inline code to codeblocks for the new features doc guideline
Fix GitLab import button
Rename behaviour to behavior in bug issue template for consistency
Convert datetime coffeescript spec to ES6
Align add button on repository view
Update CHANGELOG with 8.11.4 entries.
removed null return - renamed 'placeTop' to 'placeProfileAvatarsToTop'
Refactor Ci::Build#raw_trace
Move CHANGELOG entry to a proper version
Change widths of content in MR pipeline tab
...
Conflicts:
lib/gitlab/ci/config/node/jobs.rb
* upstream/master: (289 commits)
Fix a typo
Change minimum Unicorns required to two
Update memory requirements
Change the inline code to codeblocks for the new features doc guideline
Update CHANGELOG with 8.11.4 entries.
removed null return - renamed 'placeTop' to 'placeProfileAvatarsToTop'
Change widths of content in MR pipeline tab
Add curve to generic commit status pipeline
Rubocop syntax 2.3
Some minor updates for upgrade guides for 8.12.
Remove inconsistent font weight for sidebar's labels
Replace play icon font with svg
Project tools visibility level
Added todo filter tests
Fixed project filtering
Review changes, simplified dropdown init
Removed select2 from todos feature spec
Removed inline JS and improved dropdown labels
Added type and action dropdowns, need to finalize by removing all inline and polishing off the selected dropdown states
Completed project filter dropdown, still need to move it from inline to ProjectSelect.js (or different)
...
Enable Rubocop cops that check access modifiers
## What does this MR do?
This MR enables Rubocop cops that detect methods that should be restricted but are the part of public API because of access modifiers used improperly.
This also fixes existing offenses.
## Why was this MR needed?
Some method in our codebase are public instead of being private because it is sometimes difficult to get it right without static analysis.
## What are the relevant issue numbers?
See #17478Closes#17372
See merge request !5014
* master: (321 commits)
Fix the Sentry spam from CSP violations by disabling it.
Limit git rev-list output count to one in forced push check
Ensure Owners are included in the scope for authorized_projects
Fix alignment of icons on project page
Fix ci_status_helper_spec to look for new SVGs
use 2.0.5, actually (2.0.4 was a bad release)
upgrade rouge to 2.0.4
Fix help page paths to make sure shortcuts and the UI help page work.
fixes an issue cause by a bad merge
Vertically align status icon within table
Add new icons for every CI status
Add global style for running icon
Align running icon in merge request
Add new running icon; add a bunch of styles to get svg to match existing fa icons
Improve code design
Fix broken builds_for_ref
Move when tests before to make it no conflict with manual-actions
Use value of `yaml_variables` and `when` from config_processor if undefined
Add CHANGELOG entry
CHANGELOG item
...
Conflicts:
lib/ci/gitlab_ci_yaml_processor.rb
spec/lib/ci/gitlab_ci_yaml_processor_spec.rb
This enables following cops:
Check for useless access modifiers
Lint/UselessAccessModifier
Checks for attempts to use `private` or `protected` to set the
visibility of a class method, which does not work.
Lint/IneffectiveAccessModifier
This also disables two false possitives in concerns.
We assume that when someone adds a key for the configuration entry, but
does not provide a valid value, which causes entry to be `nil`, then
entry should be considered as the undefined one. We also assume this is
semantically correct, this is also backwards compatible with legacy CI
config processor.
See issue #18775 for more details.
* master: (345 commits)
use rails root join
fixed a couple of errors spotted in production
Fix RangeError exceptions when referring to issues or merge requests outside of max database values
Fix bug in `WikiLinkFilter`.
Small frontend code fixes and restore 8a2d88f commit
Warn about admin privilege to disable GitHub Webhooks
Listing GH Webhooks doesn't stop import process for non GH admin users
fixup! updated docs for api endpoint award emoji
Update CHANGELOG
Ensure Todos counters doesn't count Todos for projects pending delete
Add endpoints for award emoji on notes
Sort API endpoints and implement feedback
Add endpoints for Award Emoji
Fixed issue with assignee dropdown not selecting correctly
Removed update method Re-structured controller spec Renamed issuable param to issuable_id
Fix clibpoard buttons on "Check out branch" modal.
Track method call times/counts as a single metric
Cache todo counters (pending/done)
Fix a 'wrong number of arguments' error
Added missing mount point for Sidekiq Metrics API, after it got lost on rebase.
...
* master: (189 commits)
Update CHANGELOG for !4659
Center the header logo for all Devise emails
Add previews for all customized Devise emails
Customize the Devise `unlock_instructions` email
Customize the Devise `reset_password_instructions` email
Customize the Devise `password_change` emails
Use gitlab-git 10.2.0
Use Git cached counters on project show page
Fix indentation scss-lint errors
Added title attribute to enties in tree view Closes#18353
Banzai::Filter::ExternalLinkFilter use XPath
Reduce queries in IssueReferenceFilter
Use gitlab_git 10.1.4
Fixed ordering in Project.find_with_namespace
Fix images in emails
Banzai::Filter::UploadLinkFilter use XPath
Turn Group#owners into a has_many association
Make project_id nullable
...
This follows a standard `ActiveModel` pattern of creating a custom
validators. We use `ActiveModel::EachValidator` here that reuses methods
provided by `LegacyValidationHelpers`.
We will remove `LegacyValidationHelpers` on some point in the future, at
the later stages of CI configuration refactoring. It may be possible
to rewrite custom validators to use format like:
`validates :config, array_of: String`
Currently the RegexpError can be raised when processing next stage which leads to 500 in different places of code base.
This adds early check that regexps used in only and except are valid.
* master: (285 commits)
Bump recaptcha gem to 3.0.0 to remove deprecated stoken support
Load knapsack in Rakefile only when is bundled
Add License Finder information to contribution acceptance criteria.
Add LGPLv2 to license whiltelist
Instrument `RepositoryCheck::SingleRepositoryWorker` manually
Bump nokogiri to 1.6.8
Fix alignment of wiki top area
Update charcoal theme colors
Update nav link font size and spacing; fix hamburger icon
Fix control btn position
Remove todos count tests in nav
Test impersonation using img data attribute instead of username
Implement compact side nav
Fix knapsack for master
Align links and tabs
Add scrolling tabs to code subnav
Finish styling sub nav
Updated colors
Fixed failing tests
CHANGELOG item
...
Conflicts:
lib/gitlab/ci/config.rb
spec/lib/gitlab/ci/config_spec.rb
As for now, we keep this class inside a oryginal config processor class.
We will move implementation to this class and delegate to it from
current config processor.
After original gitlab ci yaml processor not longer has relevant
impelemntation we will replace it with new configuration class.
This changes the format of metadata to handle paths, that may contain
whitespace characters, new line characters and non-UTF-8 characters.
Now those paths along with metadata in JSON format are stored as
length-prefixed strings (uint32 prefix).
Metadata file has a custom format:
1. First string field is metadata version field (string)
2. Second string field is metadata errors field (JSON strong)
3. All subsequent fields is pair of path (string) and path metadata
in JSON format.
Path's metadata contains all fields that where possible to extract from
ZIP archive like date of modification, CRC, compressed size,
uncompressed size and comment.
Lin Jen-Shin
Grzegorz Bizon
Kamil Trzcinski
Rémy Coutable
Robert Speicher