Commit Graph

216 Commits

Author SHA1 Message Date
gfyoung 7ec8af5017 Enable even more frozen string for lib/gitlab
Enables frozen string for the following:

* lib/gitlab/hook_data/**/*.rb
* lib/gitlab/i18n/**/*.rb
* lib/gitlab/import/**/*.rb
* lib/gitlab/import_export/**/*.rb
* lib/gitlab/kubernetes/**/*.rb
* lib/gitlab/legacy_github_import/**/*.rb
* lib/gitlab/manifest_import/**/*.rb
* lib/gitlab/metrics/**/*.rb
* lib/gitlab/middleware/**/*.rb

Partially addresses gitlab-org/gitlab-ce#47424.
2018-11-16 17:41:14 -08:00
William George 1b153d497b Make getting a user by the username case insensitive 2018-10-18 09:06:44 +00:00
Mark Chao bec7f06097 Ignore irrelevant sql commands in metrics
Fix #51005
2018-09-05 09:10:19 +08:00
Yorick Peterse 81d8ddf8ca
Add a 10 ms bucket for SQL timings
This allows us to calculate quantiles of SQL timings more accurately,
instead of a lot of timings being assigned to the 50 ms bucket.
2018-07-10 15:59:30 +02:00
Lin Jen-Shin 9286f5b934 Use stable gitlab-styles and eliminate offenses 2018-07-09 21:13:08 +08:00
Lin Jen-Shin 4ee08b77bc Updates from `rubocop -a` 2018-07-09 21:13:08 +08:00
Ben Kochie 78a9991543
Cleanup ruby sampler metrics
* Use a simple counter for sampler duration instead of a histogram.
* Use a counter to collect GC time.
* Remove unused objects metric.
* Cleanup metric names to match Prometheus conventions.
* Prefix generic GC stats with `gc_stat`.
* Include worker label on memory and file descriptor metrics.
2018-06-22 14:12:22 +02:00
Stan Hu 88ceef87e3 Merge branch 'limit-metrics-content-type' into 'master'
Limit the action suffixes in transaction metrics

See merge request gitlab-org/gitlab-ce!20032
2018-06-20 16:34:42 +00:00
Yorick Peterse d527b46802
Remove remaining traces of the Allocations Gem
In MR https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15834 we
removed use of the data produced by the Allocations Gem. However, we
never removed the code that just enables tracking of allocations. In
this commit we remove all remaining traces of this Gem.
2018-06-20 15:11:10 +02:00
Yorick Peterse d73e68deb5
Limit the action suffixes in transaction metrics
There seem to be a lot of cases where the suffix of an action (e.g.
".html") is set to bogus data, such as "*/*" or entire URLs. This can
increase cardinality of our metrics, and isn't very useful for
monitoring and filtering. To work around this, we enforce a whitelist
containing a few suffixes we actually care about. Suffixes not supported
will be grouped under the action without a suffix. This means that a
request to "FooController#bar.jpeg" will be assigned to
"FooController#bar".
2018-06-20 14:35:04 +02:00
Yorick Peterse 61ae9c4c5b
Adjust SQL and transaction Prometheus buckets
This allows us to better calculate Apdex scores, instead of having to
use the 0.1 and 1.0 buckets.
2018-06-11 19:18:02 +02:00
Yorick Peterse 33af33d7cb
Fix GPM content types for Doorkeeper
Certain controllers (e.g. Doorkeeper::TokensController) don't expose the
method "request_format". This commit changes
Gitlab::Metrics::WebTransaction so we don't rely on this method, instead
using the underlying code this method uses.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/46412
2018-05-16 12:53:54 +02:00
Yorick Peterse c003337729
Fix setting Gitlab metrics content types
The method "content_type" on a controller does not always return the
correct content type. On the other hand, the method "request_format"
does _and_ immediately returns a Symbol (e.g. :json) instead of a
mime-type name (e.g. application/json). With these changes metrics
should again report their action names correctly.

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/3499
2018-05-15 17:38:39 +02:00
Stan Hu 204af2e101 Support resetting of Prometheus metrics between test runs
Adding the :prometheus tag to an rspec test will clear out
memory-mapped files and reset the registry.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/39968
2018-05-09 14:33:42 -07:00
Stan Hu 05e1cbc4ca Move Sidekiq exporter logs to log/sidekiq_exporter.log
The Sidekiq exporter logs were mixing with the normal Sidekiq logs. In order
to support structured logging in Sidekiq, we either need to split this data
out or convert the exporter to produce structured logs. Since Sidekiq job
processing is fundamentally different information from Web server traffic,
it seems cleaner to move the metrics traffic into a separate file, where they
can be parsed by a different filter if needed.

Relates to #20060
2018-04-02 08:58:17 -07:00
Stan Hu 02d2a789d2 Fix "A copy of Gitlab::Metrics::Methods" have been removed error
Closes #44365
2018-03-16 12:05:39 -07:00
Stan Hu 3b3615f0b7 Guard against nil Sidekiq metrics server
Closes #41351
2018-02-20 21:47:55 -08:00
Pawel Chojnacki 50c9ba43f8 Only use features for events 2018-02-20 14:06:12 +01:00
Pawel Chojnacki 93c0a168ab Put all event metrics exposed to prometheus behind a feature flag 2018-02-20 13:29:05 +01:00
Mario de la Ossa eaada9d706 use Gitlab::UserSettings directly as a singleton instead of including/extending it 2018-02-02 18:39:55 +00:00
Takuya Noguchi 2b6307f6ad Enable RuboCop Style/RegexpLiteral 2018-02-01 02:06:07 +09:00
Pawel Chojnacki d4c768ce09 Rename Concern -> Methods 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 098cf7d257 add comment aabout hotspots 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 75b49298f7 remove unused (?) variables 2018-01-29 15:13:05 +01:00
Pawel Chojnacki ffac08d290 Use class variables for metrics provider cache 2018-01-29 15:13:05 +01:00
Pawel Chojnacki a760445f9b fix typo in the buckets 2018-01-29 15:13:05 +01:00
Pawel Chojnacki b1ba8eb9a2 Do not use include Gitlab::CurrentSettings unnecessarily 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 1ac96603e1 Let mutex be initialized only once 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 05373c2490 Fix wrong histograam buckets 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 1c403459ca Add missing feature gate for cpu metrics 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 938d9ffe40 Refactor metrics to use metrics dsl notation 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 7d716cc849 Convert InfluxDB to concern. Fix uninitialized metrics when metrics code is inherited. 2018-01-29 15:13:04 +01:00
Pawel Chojnacki ea6196d490 Move prometheus metrics to module. Use class instance variables for metrics and metrics cache in metrics concern 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 27359dbf22 use class variable for Mutex instead of Const 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 83ae172145 Cleanup rspec 2018-01-29 15:13:04 +01:00
Pawel Chojnacki f5383578d9 NullMetric as a singleton 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 3e898be8aa Avoid cascading locking 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 6ff9f028fd Migrate and reduce the buckets of Gitaly metrics
/cc @bjk-gitlab
/cc @zj
2018-01-29 15:13:04 +01:00
Pawel Chojnacki 67fcd0610c Test if feature is respected 2018-01-29 15:13:04 +01:00
Pawel Chojnacki e3fe65543b Add Tests for Metrics::Concern 2018-01-29 15:13:04 +01:00
Pawel Chojnacki d7edc4b46b Use Global Mutex, and class instance variables using
```
          metric = @_metrics_provider_cache&.[](name)
          return metric if metric
``
acccessor that only slightly slower (<20ns) than direct class variable access
2018-01-29 15:13:03 +01:00
Pawel Chojnacki 087c9a5e8f Fix rubocop warnings 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 0325c23732 Fix tests 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 91a991367e Convert active record 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 66c2a3a83e Fix helper methods and document metric builder options 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 5328584b1e Reduce cardinality and convert some metrics to new scheme 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 62ef34cb40 Put View instrumentation and transaction memory use behind feature 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 66c1acba0b Fix code after refactoring 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 53f818fdac Refactor method concern to use hashes 2018-01-29 15:13:03 +01:00
Pawel Chojnacki e56b3d180d Refactor transaction metrics 2018-01-29 15:13:03 +01:00