Commit Graph

396 Commits

Author SHA1 Message Date
Grzegorz Bizon 5b7f211cbb Add new CI config entry that holds jobs definition 2016-07-05 13:35:50 +02:00
Grzegorz Bizon 9686a4f2b2 Remove code creating job hash from legacy CI config 2016-07-05 13:19:22 +02:00
Grzegorz Bizon cf53d79873 Extract jobs config to separate key in config hash 2016-07-05 13:07:03 +02:00
Grzegorz Bizon bfad4c61f1 Add minor improvements in readability in CI config 2016-07-04 18:36:06 +02:00
Grzegorz Bizon eb151e77ff Extract CI configuration entry node factory method 2016-07-04 11:29:59 +02:00
Grzegorz Bizon b4f03e8b1e Improve description of CI types node and in specs 2016-06-30 12:59:17 +02:00
Grzegorz Bizon 7ef11ce3de Explicitly define entry node class in new CI config 2016-06-29 12:20:45 +02:00
Grzegorz Bizon c8c930f3ff Add CI config entry validator for allowed keys 2016-06-29 11:01:26 +02:00
Grzegorz Bizon b85d4969a9 Return compound value if CI config node is composite 2016-06-29 10:02:02 +02:00
Grzegorz Bizon 7759242ae5 Move global CI cache configuration to new CI classes 2016-06-29 09:49:46 +02:00
Grzegorz Bizon f4421817de Add global cache config entry to new CI config 2016-06-29 09:48:52 +02:00
Grzegorz Bizon 92312786f1 Add CI config entry location info to error message
This CI config entry location in configuration Hash.
2016-06-29 09:17:42 +02:00
Grzegorz Bizon 7c511c2f55 Make it possible to set parent in CI config node 2016-06-29 09:05:14 +02:00
Grzegorz Bizon c019585cb8 Validate interface only with CI node validator 2016-06-27 14:16:57 +02:00
Grzegorz Bizon 56e88b8c28 Add new ci config entry that handles cache config 2016-06-27 14:01:12 +02:00
Grzegorz Bizon ce4478ed86 Add ci config entry that represents array of paths 2016-06-24 09:54:52 +02:00
Grzegorz Bizon e017e1b629 Add ci config class that represents a boolean value 2016-06-24 09:49:54 +02:00
Grzegorz Bizon 04ece6664a Add ci config class that represents a key value 2016-06-24 08:58:09 +02:00
Grzegorz Bizon 1f320edb77 Minor refactorings in new CI configuration classes 2016-06-23 13:51:34 +02:00
Grzegorz Bizon 29b96d92c1 Move CI stages configuration to new CI config 2016-06-23 13:51:07 +02:00
Grzegorz Bizon 2240807c1a Assume that unspecified CI config is undefined
We assume that when someone adds a key for the configuration entry, but
does not provide a valid value, which causes entry to be `nil`, then
entry should be considered as the undefined one. We also assume this is
semantically correct, this is also backwards compatible with legacy CI
config processor.

See issue #18775 for more details.
2016-06-23 10:22:41 +02:00
Grzegorz Bizon bc2348f2e4 Return default config value when entry is undefined 2016-06-22 14:26:33 +02:00
Grzegorz Bizon 05ce8a1187 Handle CI environment variables in a new CI config 2016-06-22 11:22:53 +02:00
Grzegorz Bizon 04ecfca386 Rename CI config null node entry to undefined node 2016-06-22 10:44:33 +02:00
Grzegorz Bizon 97ec24f0b0 Return CI entry config as value by default 2016-06-21 13:12:58 +02:00
Grzegorz Bizon d399128955 Handle after script CI config in new classes
This also makes Script to return an array of commands instead of
concatented command, which is our current direction.
2016-06-21 13:02:14 +02:00
Grzegorz Bizon fc00c545b2 Handle CI services config in new CI config classes 2016-06-21 12:40:52 +02:00
Grzegorz Bizon cd6a2afbbb Move CI image configuration entry to new CI config 2016-06-21 12:10:13 +02:00
Grzegorz Bizon 8b550db33e Add image configuration entry to new ci config 2016-06-21 11:47:05 +02:00
Grzegorz Bizon c91298d554 Use generic type validator in new ci configuration 2016-06-21 11:43:32 +02:00
Grzegorz Bizon 9510d31b4d Merge branch 'master' into refactor/ci-config-add-entry-error
* master: (345 commits)
  use rails root join
  fixed a couple of errors spotted in production
  Fix RangeError exceptions when referring to issues or merge requests outside of max database values
  Fix bug in `WikiLinkFilter`.
  Small frontend code fixes and restore 8a2d88f commit
  Warn about admin privilege to disable GitHub Webhooks
  Listing GH Webhooks doesn't stop import process for non GH admin users
  fixup! updated docs for api endpoint award emoji
  Update CHANGELOG
  Ensure Todos counters doesn't count Todos for projects pending delete
  Add endpoints for award emoji on notes
  Sort API endpoints and implement feedback
  Add endpoints for Award Emoji
  Fixed issue with assignee dropdown not selecting correctly
  Removed update method Re-structured controller spec Renamed issuable param to issuable_id
  Fix clibpoard buttons on "Check out branch" modal.
  Track method call times/counts as a single metric
  Cache todo counters (pending/done)
  Fix a 'wrong number of arguments' error
  Added missing mount point for Sidekiq Metrics API, after it got lost on rebase.
  ...
2016-06-20 10:38:46 +02:00
Grzegorz Bizon 44b00a1ebb Extract CI entry config hash validation to validator 2016-06-17 15:02:39 +02:00
Grzegorz Bizon 2a87a55f87 Merge branch 'master' into refactor/ci-config-add-entry-error
* master: (189 commits)
  Update CHANGELOG for !4659
  Center the header logo for all Devise emails
  Add previews for all customized Devise emails
  Customize the Devise `unlock_instructions` email
  Customize the Devise `reset_password_instructions` email
  Customize the Devise `password_change` emails
  Use gitlab-git 10.2.0
  Use Git cached counters on project show page
  Fix indentation scss-lint errors
  Added title attribute to enties in tree view Closes #18353
  Banzai::Filter::ExternalLinkFilter use XPath
  Reduce queries in IssueReferenceFilter
  Use gitlab_git 10.1.4
  Fixed ordering in Project.find_with_namespace
  Fix images in emails
  Banzai::Filter::UploadLinkFilter use XPath
  Turn Group#owners into a has_many association
  Make project_id nullable
  ...
2016-06-17 12:25:43 +02:00
Grzegorz Bizon d9ca84015c Add first custom validator for new ci config
This follows a standard `ActiveModel` pattern of creating a custom
validators. We use `ActiveModel::EachValidator` here that reuses methods
provided by `LegacyValidationHelpers`.

We will remove `LegacyValidationHelpers` on some point in the future, at
the later stages of CI configuration refactoring. It may be possible
to rewrite custom validators to use format like:

`validates :config, array_of: String`
2016-06-17 12:06:48 +02:00
Grzegorz Bizon a9bd16bd0a Rename legacy validation helpers for new ci config 2016-06-17 11:55:55 +02:00
Kamil Trzcinski aef6214c42 Validate only and except regexp
Currently the RegexpError can be raised when processing next stage which leads to 500 in different places of code base.
This adds early check that regexps used in only and except are valid.
2016-06-17 11:52:22 +02:00
Grzegorz Bizon 002e6ed1f0 Improve CI config entries validations prototype 2016-06-17 11:23:16 +02:00
Grzegorz Bizon 95520dfc72 Add prototype of CI config node validator
This makes use of `ActiveModel::Validations` encapsulated in a separate
class that is accessible from a node object.
2016-06-16 15:46:03 +02:00
Grzegorz Bizon 76aea978c6 Add class that encapsulates error in new Ci config 2016-06-15 14:09:21 +02:00
Kamil Trzcinski 006b650988 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	db/schema.rb
2016-06-14 16:19:29 +02:00
Kamil Trzcinski 975e388751 Merge remote-tracking branch 'origin/master' into environments-and-deployments
# Conflicts:
#	lib/ci/gitlab_ci_yaml_processor.rb
2016-06-14 14:53:10 +02:00
Kamil Trzcinski 3f5819fb7a Merge remote-tracking branch 'origin/master' into artifacts-expire-date 2016-06-14 14:49:59 +02:00
Grzegorz Bizon 30e946ce8a Validate ci config entry value before processing nodes 2016-06-14 11:28:20 +02:00
Grzegorz Bizon 59eeec3ff8 Make method that composes ci config entry private 2016-06-14 10:48:51 +02:00
Grzegorz Bizon 11c0d02283 Simplify ci config node factory 2016-06-13 09:58:35 +02:00
Grzegorz Bizon 7c8f3b0cfc Duplicate CI config node factory on class level 2016-06-13 08:50:12 +02:00
Grzegorz Bizon cc373a3550 Add factory for fabricating new ci config nodes 2016-06-10 14:01:07 +02:00
Grzegorz Bizon 5abfc7fa71 Define ci entry accessor instead of method_missing 2016-06-10 11:32:49 +02:00
Grzegorz Bizon 12080ba150 Simplify new ci config entry class interface 2016-06-10 11:26:03 +02:00
Grzegorz Bizon 828a15bccd Rename method used to allow node in Ci config 2016-06-10 10:49:47 +02:00
Grzegorz Bizon d7e1251161 Rename method that returns allowed nodes in Ci config 2016-06-09 14:59:59 +02:00
Grzegorz Bizon 99ee39bf6c Add comments to new CI config classes and modules 2016-06-09 13:15:54 +02:00
Grzegorz Bizon 20ccd4465b Do not require Ci config node to have a hash value 2016-06-09 12:35:24 +02:00
Grzegorz Bizon 6a319fd287 Make it possible configure Ci entry description 2016-06-09 10:53:56 +02:00
Grzegorz Bizon d9d5042fd9 Extract method that composes new Ci config entry 2016-06-09 10:28:44 +02:00
Grzegorz Bizon 33cd090b93 Move new Ci config configurable DSL to concern 2016-06-09 10:08:49 +02:00
Grzegorz Bizon 48a59c1a8b Rename BeforeScript to Script in new Ci config 2016-06-08 13:22:39 +02:00
Grzegorz Bizon 5065612a0a Add minor improvements in new Ci config design 2016-06-08 13:01:44 +02:00
Grzegorz Bizon 87fe50f2a0 Delegate Ci config entry value to single method 2016-06-08 12:32:56 +02:00
Grzegorz Bizon 2b48da22ca Merge branch 'master' into refactor/ci-config-add-global-entry
* master: (285 commits)
  Bump recaptcha gem to 3.0.0 to remove deprecated stoken support
  Load knapsack in Rakefile only when is bundled
  Add License Finder information to contribution acceptance criteria.
  Add LGPLv2 to license whiltelist
  Instrument `RepositoryCheck::SingleRepositoryWorker` manually
  Bump nokogiri to 1.6.8
  Fix alignment of wiki top area
  Update charcoal theme colors
  Update nav link font size and spacing; fix hamburger icon
  Fix control btn position
  Remove todos count tests in nav
  Test impersonation using img data attribute instead of username
  Implement compact side nav
  Fix knapsack for master
  Align links and tabs
  Add scrolling tabs to code subnav
  Finish styling sub nav
  Updated colors
  Fixed failing tests
  CHANGELOG item
  ...

Conflicts:
	lib/gitlab/ci/config.rb
	spec/lib/gitlab/ci/config_spec.rb
2016-06-08 11:38:56 +02:00
Grzegorz Bizon 70bda3e89b Implement script in Ci config and use in legacy one 2016-06-07 14:23:47 +02:00
Grzegorz Bizon c2d6d61dac Add DSL for adding nodes in Ci config interface 2016-06-07 13:20:07 +02:00
Grzegorz Bizon df25c19699 Use Ci config validation helpers only where needed 2016-06-07 12:58:32 +02:00
Grzegorz Bizon 5ef104df59 Improve Ci config loader by changing method signature 2016-06-07 12:53:46 +02:00
Grzegorz Bizon 6bd67f5212 Do not process new Ci config entry when invalid 2016-06-07 12:48:26 +02:00
Grzegorz Bizon e8f995ef26 Pass root Ci config entry to each subsequent entry 2016-06-07 12:13:22 +02:00
Grzegorz Bizon 69a3755c5a Add Ci config entry that implements Null Object 2016-06-07 11:58:02 +02:00
Grzegorz Bizon b95c60a071 Do not process Ci config node when node is a leaf 2016-06-07 11:26:39 +02:00
Grzegorz Bizon fa097c678c Remove duplicated exception in Ci config
This is a temporary refactoring stub, that is planned to be removed
after removing legacy config processor.
2016-06-07 10:26:38 +02:00
Grzegorz Bizon 940763e0e7 Use CI config errors from new processor in legacy one 2016-06-06 20:56:58 +02:00
Grzegorz Bizon a3c0745514 Collect errors from all nodes in new CI config 2016-06-06 11:59:25 +02:00
Grzegorz Bizon 6dbd1c86a8 Validate new before script CI configuration entry 2016-06-06 11:33:45 +02:00
Grzegorz Bizon 251dd571df Extract CI config validation helpers to mixin 2016-06-06 11:05:15 +02:00
Grzegorz Bizon 8048dcc8e6 Implement CI configuration nodes tree processing 2016-06-06 10:43:11 +02:00
Grzegorz Bizon 7f2f683eeb Rename ci config module that holds nodes to Node 2016-06-06 09:24:16 +02:00
Grzegorz Bizon 6609589b93 Add ci config global and before_script entries 2016-06-06 09:05:00 +02:00
Grzegorz Bizon 23030439c2 Rename class that loads CI configuration to Loader 2016-06-06 08:20:55 +02:00
Grzegorz Bizon d2b708ac43 Extract CI config YAML parser to a separate class
With this approach it would be easier to add different sources of
configuration, that we do not necessairly have to be in YAML format.
2016-06-03 21:10:50 +02:00
Grzegorz Bizon d501850e05 Add gitlab ci configuration class that holds hash
As for now, we keep this class inside a oryginal config processor class.
We will move implementation to this class and delegate to it from
current config processor.

After original gitlab ci yaml processor not longer has relevant
impelemntation we will replace it with new configuration class.
2016-06-03 14:20:34 +02:00
Grzegorz Bizon 0ea017d997 Enable Style/WhileUntilDo rubocop style cop
See #17478
2016-05-30 11:37:14 +02:00
Kamil Trzcinski 9b0f57781e Add method that calculates total size for artifacts subfolder 2016-01-20 21:55:13 +01:00
Grzegorz Bizon be764a3a20 Minor improvements in build arfifacts browser
Added also a `Gitlab::Ci::Build::Artifacts::Metadata::ParserError`
exception class.
2016-01-14 12:48:17 +01:00
Grzegorz Bizon 0d6e7b9d3d Use Hash to store paths and entries metadata in artifacts browser 2016-01-14 12:48:17 +01:00
Grzegorz Bizon ad2b0358e0 Improve readability of artifacts `Metadata` related code 2016-01-14 12:48:17 +01:00
Grzegorz Bizon 6b0a43aff3 Improve readability of artifacts browser `Entry` related code 2016-01-14 12:48:17 +01:00
Grzegorz Bizon 3f0c18f80e Simplify encoding related implementation in artifacts metadata 2016-01-14 12:48:17 +01:00
Grzegorz Bizon a9783c439b Make encoding of paths returned by metadata consistent (UTF-8) 2016-01-14 12:48:16 +01:00
Grzegorz Bizon 487b0a026f Improvements, readability for artifacts browser 2016-01-14 12:48:16 +01:00
Grzegorz Bizon 2be76355ca Support only valid UTF-8 paths in build artifacts browser 2016-01-14 12:48:16 +01:00
Grzegorz Bizon ffee05c242 Improve invalid build artifacts metadata path matcher 2016-01-14 12:48:16 +01:00
Grzegorz Bizon 09a4a5aff8 Render only valid paths in artifacts metadata
In this version we will support only relative paths in artifacts
metadata. Support for absolute paths will be introduced later.
2016-01-14 12:48:16 +01:00
Grzegorz Bizon 61fb47a432 Simplify implementation of build artifacts browser (refactoring) 2016-01-14 12:48:15 +01:00
Grzegorz Bizon 387b27813d Change format of artifacts metadata from text to binary 0.0.1
This changes the format of metadata to handle paths, that may contain
whitespace characters, new line characters and non-UTF-8 characters.

Now those paths along with metadata in JSON format are stored as
length-prefixed strings (uint32 prefix).

Metadata file has a custom format:

1.   First string field is metadata version field (string)
2.   Second string field is metadata errors field (JSON strong)
3.   All subsequent fields is pair of path (string) and path metadata
     in JSON format.

Path's metadata contains all fields that where possible to extract from
ZIP archive like date of modification, CRC, compressed size,
uncompressed size and comment.
2016-01-14 12:48:15 +01:00
Grzegorz Bizon 1b1793c253 Show file size in artifacts browser using metadata 2016-01-14 12:48:15 +01:00
Grzegorz Bizon a5e1905d28 Render 404 when artifacts path is invalid 2016-01-14 12:48:15 +01:00
Grzegorz Bizon a7f99b67a0 Extract artifacts metadata implementation to separate class 2016-01-14 12:48:15 +01:00