Commit Graph

313 Commits

Author SHA1 Message Date
Yorick Peterse bd3699dc6e
Removed duplicate functions from utils.sh
This change was made in EE in MR
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14802, but this
was not backported to CE.
2019-08-09 15:34:06 +02:00
Stan Hu a74396dcc5 Add Gitaly and Rugged call timing in Sidekiq logs
This will help identify Sidekiq jobs that invoke excessive number of
filesystem access.

The timing data is stored in `RequestStore`, but this is only active
within the middleware and is not directly accessible to the Sidekiq
logger. However, it is possible for the middleware to modify the job
hash to pass this data along to the logger.
2019-08-09 01:08:32 -07:00
Winnie Hellmann 4541b44c36 Add script for frontend test file coverage 2019-08-08 07:31:43 +00:00
Rémy Coutable 8f501a74bd
Remove the wait_for_review_app_to_be_accessible step from review-deploy
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-07-30 19:13:43 +02:00
Rémy Coutable 9870e86f6c
Use the --atomic flag when deploying Review Apps
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-07-30 09:25:28 +02:00
Yorick Peterse d7a4c1d2a5
Backport EE changes made to the scripts/ directory
This backport changes made by EE to the files in the scripts/ directory.
This comes with a few changes to some scripts to make them work in the
single codebase setup.
2019-07-29 13:53:20 +02:00
Douglas Barbosa Alexandre aa9db9c869 Merge branch '54478-table_exists-not-compatible-with-rails-5-1' into 'master'
Resolve "table_exists? not compatible with Rails 5.1"

Closes #54478

See merge request gitlab-org/gitlab-ce!30832
2019-07-25 15:05:07 +00:00
Nick Thomas 72c1680424 Merge branch 'qa-stop-printing-dots-while-retrieving-pod-names-in-review-deploy' into 'master'
Stop printing dots while retrieving pod in review-apps.sh

See merge request gitlab-org/gitlab-ce!31062
2019-07-25 14:19:27 +00:00
Heinrich Lee Yu 2b3d00a778 Remove unneeded monkey-patch
Changes all calls to data_source_exists? to table_exists?
since that is the intent of these calls
2019-07-25 12:21:38 +08:00
Sean McGivern c02c83fa9b Merge branch '64998-increase-helm-deploy-command-timeout' into 'master'
Increase the Review App deploy command timeout to 15 minutes

Closes #64998

See merge request gitlab-org/gitlab-ce!31061
2019-07-24 14:29:21 +00:00
Sean McGivern 562db3a36e Merge branch 'sh-add-rugged-to-peek' into 'master'
Add Rugged calls to performance bar

See merge request gitlab-org/gitlab-ce!30983
2019-07-24 12:24:12 +00:00
Lin Jen-Shin aef27e6acc Merge branch '64996-fix-review-app-debugging' into 'master'
Resolve "Review App debugging sometimes fail because multiple Unicorn pods exist"

Closes #64996

See merge request gitlab-org/gitlab-ce!31058
2019-07-24 09:03:11 +00:00
Stan Hu 291df05e43 Add Rugged calls to performance bar
This will help diagnose the source of excessive I/O from Rugged
calls. To implement this, we need to obtain the full list of arguments
sent to each request method.
2019-07-23 21:38:05 -07:00
Rémy Coutable 28d9e87d40
Stop printing dots while retrieving pod in review-apps.sh
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-07-23 18:30:47 +02:00
Rémy Coutable b4f695da6b
Increase the Review App deploy timeout to 15 minutes
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-07-23 17:53:21 +02:00
Nick Thomas 4aa76dddec
Remove dead MySQL code
None of this code can be reached any more, so it can all be removed
2019-07-23 16:53:03 +01:00
Rémy Coutable dccf643c42
Fix Review App debugging
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-07-23 17:11:13 +02:00
Peter Dave Hello b2f143597c Extend yamllint test to changelogs
cc !30979
2019-07-23 12:23:51 +00:00
Stan Hu 1136c0c8e9 Add Rugged calls and duration to API and Rails logs
This adds `rugged_duration_ms` and `rugged_calls` fields to
`api_json.log` and `production_json.log`. This will make it easier to
identify performance issues caused by excessive I/O.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64676
2019-07-18 06:53:39 -07:00
Desiree Chevalier a1bee1a09d Generate parallel spec reports
Creates reports for parallel specs and collates them into a single
HTML report and displays results on the merge request.
2019-07-17 10:12:56 -04:00
Peter Leitzen 05b96f0b67 Fix link pointing to the documentation styleguide
Using the old link redirects to
https://docs.gitlab.com/ee/development/documentation/index.html
which does not show the mentioned documentation.
2019-07-16 06:46:07 +00:00
Grzegorz Bizon f69232d506 Merge branch '64356-pin-external-dns-version' into 'master'
ReviewApps: CI pin external-dns to 2.x version

Closes #64356

See merge request gitlab-org/gitlab-ce!30577
2019-07-12 15:24:04 +00:00
Jason Plum 3c82d9fc8f ReviewApps: CI pin external-dns to 2.x version
Pin the version of deployed external-dns to `2.y.z`, using SemVer
matching. (`^2.2.1`)

Update the properties for the service account keys per
*Upgrading to 2.0.0* of the `stable/external-dns` README.

Add `aws.batchChangeSize=400` to combat messages about RDATA being
too large.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64356
2019-07-12 09:53:41 -04:00
Marcia Ramos cb0edce20e Merge branch 'rename-gitlab-docs-repo' into 'master'
Move the gitlab-docs project under gitlab-org

See merge request gitlab-org/gitlab-ce!30620
2019-07-12 11:43:30 +00:00
Aleksei Lipniagov 4085428ebd Gather memory usage data in tests
Log memory stats after running each spec file and compile the report.
2019-07-12 10:44:17 +00:00
Achilleas Pipinellis 99f2c986b5
Move the gitlab-docs project under gitlab-org
Search and replace gitlab-com/gitlab-docs with gitlab-org/gitlab-docs.
For https://gitlab.com/gitlab-com/gitlab-docs/issues/310
2019-07-11 15:24:33 +03:00
Mike Lewis 77ff2249e4 Fix docs URL in trigger-build-docs puts text 2019-07-08 15:44:50 +00:00
Rémy Coutable 29dbac2e12
Add resources requests and limits for all Review Apps resources
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-07-04 18:14:11 +02:00
Aleksei Lipniagov f44719f39c Get memory data after booting app in production
Preload the app in `production` env, hit it with a single request, and
gather total gem memory consumption data using `derailed exec perf:mem`
from `derailed_benchmarks`. Present the result as MR metrics.
2019-06-28 20:12:50 +03:00
Rémy Coutable 7c38c0b4b2 Revert "Merge branch '62722-boot-app-in-ci-and-check-memory' into 'master'"
This reverts merge request !30031
2019-06-28 15:55:54 +00:00
Aleksei Lipniagov fb1d433703 Get memory data after booting app in production
Preload the app in `production` env, hit it with a single request, and
gather total gem memory consumption data using `derailed exec perf:mem`
from `derailed_benchmarks`. Present the result as MR metrics.
2019-06-28 13:49:57 +03:00
Lin Jen-Shin 51011d1b2c Merge branch 'improve-review-apps-cleanup-when-previous-deployment-failed' into 'master'
Improve Review Apps cleanup when previous deployment failed by only issuing an `helm delete` command

Closes #63639 and #62161

See merge request gitlab-org/gitlab-ce!28661
2019-06-25 14:00:47 +00:00
Lin Jen-Shin 938db91322 Merge branch '62735-for-stable-branches-build-cng-images-from-corresponding-stable-branches-in-cng-mirror-project' into 'master'
Pass branch name to the CNG pipeline for stable branches

See merge request gitlab-org/gitlab-ce!29617
2019-06-25 11:18:54 +00:00
Rémy Coutable e266911019
Install Helm on dedicated nodes
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-06-25 11:09:06 +02:00
Rémy Coutable a90f12f77d
Let Helm handle the cleanup and print logs when deployment fails
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-06-25 11:09:06 +02:00
Qingyu Zhao 940fcc9ff2 Generate gem size metrics in memory CI
This includes several changes:
  - Rename memory-static to generate-gems-size-metrics-static
  - Rename memory-static-objects to generate-gems-memory-metrics-static
  - Change generate-gems-size-metrics-static interface. The script now
    expect `bundle exec derailed bundle:mem` output as its input. The
    script output to stdout, or stderr for error message.
  - Change generate-gems-memory-metrics-static interface. The script now
    expect `bundle exec derailed bundle:objects` output as its input.
    The script output to stdout, or stderr for error message.
  - Generate gem size metrics. Script generate-gems-size-metrics-static
    extract each gem size from `bundle exec derailed bundle:mem` output.
    Save output to metrics file in format: 'gem_size_mb{name="zip"} 0.5'
2019-06-21 20:05:59 +10:00
Paul Gascou-Vaillancourt bd9774e4d4 Add watch option to test script 2019-06-19 16:05:46 +00:00
Rémy Coutable 7fd2d687db
Pass branch name to the CNG pipeline for stable branches
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-06-19 16:37:33 +02:00
Aleksei Lipniagov c3b40ae131 Run static benchmarks from 'derailed_benchmarks'
Two static memory benchmarks will be included in our CI pipeline. It
will load gems from the Gemfile and check the amount of RAM consumed
as well as the number of objects allocated and retained.
Aggregated values will be included as 'metrics' into MRs while full
reports will be downloadable as job artifacts.
2019-06-19 12:16:44 +03:00
Bob Van Landuyt e6059b3d8d Merge branch 'zj-disable-hooks-testing' into 'master'
Disable hooks by setting an ENV var

See merge request gitlab-org/gitlab-ce!29672
2019-06-17 17:32:58 +00:00
Zeger-Jan van de Weg 679ac7f78e
Disable hooks by setting an ENV var
Prior to this change, the hooks directory got cleared. That works, but
is not the way to go about it as there's a better way. Setting the env
var this commits sets.
2019-06-14 13:10:02 +02:00
Rémy Coutable 7c1422c795 Apply suggestion to scripts/review_apps/review-apps.sh 2019-06-12 17:01:15 +00:00
Rémy Coutable 115966335b
Pass FORCE_RAILS_IMAGE_BUILDS=true to the CNG triggered pipeline
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-06-12 13:22:12 +02:00
Rémy Coutable 97037e4a11
[Review Apps] Set `global.imagePullPolicy: Always`
Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-06-12 13:22:12 +02:00
Kamil Trzciński 21c809ce22 Merge branch '62151-broken-master' into 'master'
Fix MySQL CI jobs

Closes #62156 and #62151

See merge request gitlab-org/gitlab-ce!28593
2019-05-24 15:34:54 +00:00
Clement Ho 57d9f88fd5 Merge branch '58869-unified-fe-test-script' into 'master'
Create a unified script to run Jest & Karma tests

Closes #58869

See merge request gitlab-org/gitlab-ce!27239
2019-05-23 17:32:56 +00:00
Rémy Coutable d02d3e34dd
Fix MySQL CI jobs
[run mysql]

Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-05-22 21:47:32 +02:00
Rémy Coutable 6145ddf515
Revert "Merge branch 'revert-04c3c6dd' into 'master'"
This reverts commit 744f1f2e70, reversing
changes made to c4d930e5f5.
2019-05-22 21:47:32 +02:00
Kamil Trzciński 4f20c9de6f Revert "Merge branch 'reorganize-tests-jobs-by-level' into 'master'"
This reverts merge request !27514
2019-05-22 11:45:46 +00:00
Kamil Trzciński 04c3c6dd5a Merge branch 'reorganize-tests-jobs-by-level' into 'master'
Reorganize test jobs by level

See merge request gitlab-org/gitlab-ce!27514
2019-05-22 10:21:38 +00:00