Commit Graph

75 Commits

Author SHA1 Message Date
GitLab Bot 448ff1fb83 Add latest changes from gitlab-org/gitlab@master 2020-07-22 03:09:40 +00:00
GitLab Bot 3a9076e0a4 Add latest changes from gitlab-org/gitlab@master 2020-07-16 18:09:35 +00:00
GitLab Bot 35c9eaced5 Add latest changes from gitlab-org/gitlab@master 2020-06-17 00:16:40 +00:00
GitLab Bot 711f859532 Add latest changes from gitlab-org/gitlab@master 2020-06-01 15:08:16 +00:00
GitLab Bot 374cf04cf2 Add latest changes from gitlab-org/gitlab@master 2020-05-20 12:07:52 +00:00
GitLab Bot 59accb4c47 Add latest changes from gitlab-org/gitlab@master 2020-05-20 00:08:20 +00:00
GitLab Bot 553a22402b Add latest changes from gitlab-org/gitlab@master 2020-05-19 12:08:21 +00:00
GitLab Bot bc692af988 Add latest changes from gitlab-org/gitlab@master 2020-05-09 00:09:39 +00:00
GitLab Bot 3795b229ab Add latest changes from gitlab-org/gitlab@master 2020-05-01 00:09:59 +00:00
GitLab Bot e33f87ac0f Add latest changes from gitlab-org/gitlab@master 2020-04-21 15:21:10 +00:00
GitLab Bot 874ead9c3a Add latest changes from gitlab-org/gitlab@master 2020-04-14 15:09:44 +00:00
GitLab Bot 8a7efa45c3 Add latest changes from gitlab-org/gitlab@master 2020-04-02 18:08:11 +00:00
GitLab Bot abae8f34f3 Add latest changes from gitlab-org/gitlab@master 2020-03-31 21:08:05 +00:00
GitLab Bot dddde902ac Add latest changes from gitlab-org/gitlab@master 2020-03-26 03:07:43 +00:00
GitLab Bot 074d013e1e Add latest changes from gitlab-org/gitlab@master 2020-03-23 15:09:36 +00:00
GitLab Bot d43aaf286f Add latest changes from gitlab-org/gitlab@master 2020-03-17 00:09:12 +00:00
GitLab Bot 1fa79760ad Add latest changes from gitlab-org/gitlab@master 2020-03-10 12:08:16 +00:00
GitLab Bot 7ebcead8cf Add latest changes from gitlab-org/gitlab@master 2020-03-09 18:07:59 +00:00
GitLab Bot 7b52c7cb63 Add latest changes from gitlab-org/gitlab@master 2020-03-02 18:07:42 +00:00
GitLab Bot e0fa0638a4 Add latest changes from gitlab-org/gitlab@master 2020-02-27 18:09:21 +00:00
GitLab Bot 951616a26a Add latest changes from gitlab-org/gitlab@master 2020-02-25 18:09:02 +00:00
GitLab Bot b042382bbf Add latest changes from gitlab-org/gitlab@master 2020-02-05 18:09:06 +00:00
GitLab Bot ca05512007 Add latest changes from gitlab-org/gitlab@master 2020-02-04 18:08:50 +00:00
GitLab Bot 88a0824944 Add latest changes from gitlab-org/gitlab@master 2020-02-04 12:09:00 +00:00
GitLab Bot fd3a95f07a Add latest changes from gitlab-org/gitlab@master 2020-01-31 09:08:53 +00:00
GitLab Bot 2c2dd5e36c Add latest changes from gitlab-org/gitlab@master 2020-01-24 12:09:01 +00:00
GitLab Bot 3f9e1b2611 Add latest changes from gitlab-org/gitlab@master 2020-01-23 15:08:46 +00:00
GitLab Bot aa0f0e9921 Add latest changes from gitlab-org/gitlab@master 2020-01-16 18:08:46 +00:00
GitLab Bot 511e761b41 Add latest changes from gitlab-org/gitlab@master 2020-01-03 15:08:33 +00:00
GitLab Bot ea4762d464 Add latest changes from gitlab-org/gitlab@master 2019-11-22 18:06:00 +00:00
GitLab Bot a92d6b36c2 Add latest changes from gitlab-org/gitlab@master 2019-11-05 09:21:54 +00:00
GitLab Bot 7b25b12016 Add latest changes from gitlab-org/gitlab@master 2019-11-04 18:06:28 +00:00
GitLab Bot a77db6bc47 Add latest changes from gitlab-org/gitlab@master 2019-10-28 15:05:58 +00:00
GitLab Bot 09ffaae132 Add latest changes from gitlab-org/gitlab@master 2019-10-23 12:06:18 +00:00
GitLab Bot b1bcdba89b Add latest changes from gitlab-org/gitlab@master 2019-10-22 12:06:20 +00:00
GitLab Bot c157f963db Add latest changes from gitlab-org/gitlab@master 2019-10-10 09:06:08 +00:00
GitLab Bot b082790134 Add latest changes from gitlab-org/gitlab@master 2019-09-18 18:06:14 +00:00
Qingyu Zhao eeeaebe608 Change Sidekiq monitor namespace
Move Gitlab::SidekiqMonitor to namespace Gitlab::SidekiqDaemon::Monitor
 - Class name and file name change
 - File path change to lib/gitlab/sidekiq_daemon/monitor.rb
 - Update class usage/reference in other files, including documentation
2019-09-11 00:11:54 +10:00
Andrew Newdigate 18e5362e87 Fix for histogram corruption in Sidekiq
This fixes a bug in which sidekiq histograms contain invalid labels.

See https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7729
for more details.
2019-08-30 14:04:45 +00:00
Kamil Trzciński dbd88c02d6 Put cancelled job in DeadSet
This replicates Sidekiq behavior
of pushing dead job into DeadSet.
2019-08-21 21:15:03 +02:00
Kamil Trzciński 8d17c4dae6 Properly handle `sidekiq` skip
Transform `CancelledError` into `JobRetry::Skip`
2019-08-21 13:21:55 +02:00
Kamil Trzciński c2cbfc5c4a Rework `Sidekiq::JobsThreads` into `Monitor`
This makes:
- very shallow `Middleware::Monitor` to only request tracking
  of sidekiq jobs,
- `SidekiqStatus::Monitor` to be responsible to maintain persistent
  connection to receive messages,
- `SidekiqStatus::Monitor` to always use structured logging
  and instance variables
2019-08-21 12:05:30 +02:00
Kamil Trzciński 75e2302d01 Allow to interrupt running jobs
This adds a middleware to track all threads
for running jobs.

This makes sidekiq to watch for redis-delivered notifications.

This makes be able to send notification to interrupt
running sidekiq jobs.

This does not take into account any native code,
as `Thread.raise` generates exception once the control gets
back to Ruby.

The separate measure should be taken to interrupt gRPC, shellouts,
or anything else that escapes Ruby.
2019-08-21 18:50:46 +10:00
Andrew Newdigate 790f64561a Allow measurement for Sidekiq jobs taking > 2.5s
Fix for https://gitlab.com/gitlab-org/gitlab-ce/issues/66319.
2019-08-20 13:52:25 +00:00
Stan Hu a74396dcc5 Add Gitaly and Rugged call timing in Sidekiq logs
This will help identify Sidekiq jobs that invoke excessive number of
filesystem access.

The timing data is stored in `RequestStore`, but this is only active
within the middleware and is not directly accessible to the Sidekiq
logger. However, it is possible for the middleware to modify the job
hash to pass this data along to the logger.
2019-08-09 01:08:32 -07:00
Shinya Maeda 8f14d3c140 Fix sidekiq memory killer warning message 2019-07-30 13:47:13 +07:00
Ryan Cobb cfea48dffd Adds direct monitoring for sidekiq metrics
This adds diirect monitoring for sidekiq metrics. This is done via
sidekiq middleware and a sampler to pull from sidekiqs api.
2019-07-29 11:53:12 +00:00
Shinya Maeda ad7acfaad0 Logging sidekiq worker class name in SidekiqMemoryKiller
Currently, SidekiqMemoryKiller does not feed worker class name in the
json structured logging. This commit extends the json parameter.
2019-07-25 17:49:10 +07:00
Andrew Newdigate 4f4de36cac Migrate correlation and tracing code to LabKit
This change is a fairly straightforward refactor to extract the tracing
and correlation-id code from the gitlab rails codebase into the new
LabKit-Ruby project.

The corresponding import into LabKit-Ruby was in
https://gitlab.com/gitlab-org/labkit-ruby/merge_requests/1

The code itself remains very similar for now.

Extracting it allows us to reuse it in other projects, such as
Gitaly-Ruby. This will give us the advantages of correlation-ids and
distributed tracing in that project too.
2019-04-18 09:57:16 +02:00
Nick Thomas 452f508143
Fix the last-ditch memory killer pgroup SIGKILL 2019-03-09 13:34:06 +00:00