Commit Graph

131 Commits

Author SHA1 Message Date
Mark Chao bec7f06097 Ignore irrelevant sql commands in metrics
Fix #51005
2018-09-05 09:10:19 +08:00
Ben Kochie 78a9991543
Cleanup ruby sampler metrics
* Use a simple counter for sampler duration instead of a histogram.
* Use a counter to collect GC time.
* Remove unused objects metric.
* Cleanup metric names to match Prometheus conventions.
* Prefix generic GC stats with `gc_stat`.
* Include worker label on memory and file descriptor metrics.
2018-06-22 14:12:22 +02:00
Stan Hu 88ceef87e3 Merge branch 'limit-metrics-content-type' into 'master'
Limit the action suffixes in transaction metrics

See merge request gitlab-org/gitlab-ce!20032
2018-06-20 16:34:42 +00:00
Yorick Peterse d527b46802
Remove remaining traces of the Allocations Gem
In MR https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15834 we
removed use of the data produced by the Allocations Gem. However, we
never removed the code that just enables tracking of allocations. In
this commit we remove all remaining traces of this Gem.
2018-06-20 15:11:10 +02:00
Yorick Peterse d73e68deb5
Limit the action suffixes in transaction metrics
There seem to be a lot of cases where the suffix of an action (e.g.
".html") is set to bogus data, such as "*/*" or entire URLs. This can
increase cardinality of our metrics, and isn't very useful for
monitoring and filtering. To work around this, we enforce a whitelist
containing a few suffixes we actually care about. Suffixes not supported
will be grouped under the action without a suffix. This means that a
request to "FooController#bar.jpeg" will be assigned to
"FooController#bar".
2018-06-20 14:35:04 +02:00
Yorick Peterse 33af33d7cb
Fix GPM content types for Doorkeeper
Certain controllers (e.g. Doorkeeper::TokensController) don't expose the
method "request_format". This commit changes
Gitlab::Metrics::WebTransaction so we don't rely on this method, instead
using the underlying code this method uses.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/46412
2018-05-16 12:53:54 +02:00
Yorick Peterse c003337729
Fix setting Gitlab metrics content types
The method "content_type" on a controller does not always return the
correct content type. On the other hand, the method "request_format"
does _and_ immediately returns a Symbol (e.g. :json) instead of a
mime-type name (e.g. application/json). With these changes metrics
should again report their action names correctly.

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/3499
2018-05-15 17:38:39 +02:00
Stan Hu 204af2e101 Support resetting of Prometheus metrics between test runs
Adding the :prometheus tag to an rspec test will clear out
memory-mapped files and reset the registry.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/39968
2018-05-09 14:33:42 -07:00
Stan Hu 05e1cbc4ca Move Sidekiq exporter logs to log/sidekiq_exporter.log
The Sidekiq exporter logs were mixing with the normal Sidekiq logs. In order
to support structured logging in Sidekiq, we either need to split this data
out or convert the exporter to produce structured logs. Since Sidekiq job
processing is fundamentally different information from Web server traffic,
it seems cleaner to move the metrics traffic into a separate file, where they
can be parsed by a different filter if needed.

Relates to #20060
2018-04-02 08:58:17 -07:00
Pawel Chojnacki d4c768ce09 Rename Concern -> Methods 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 2b5a639d79 use public_send where appropriate 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 7772dfb788 before -> after 2018-01-29 15:13:05 +01:00
Pawel Chojnacki f51147834e use define_method to define devine_metric_method. (couldn't resist) 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 67d4946347 Fix ruby sampler spec 2018-01-29 15:13:05 +01:00
Pawel Chojnacki 7d716cc849 Convert InfluxDB to concern. Fix uninitialized metrics when metrics code is inherited. 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 83ae172145 Cleanup rspec 2018-01-29 15:13:04 +01:00
Pawel Chojnacki f5383578d9 NullMetric as a singleton 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 3e898be8aa Avoid cascading locking 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 67fcd0610c Test if feature is respected 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 5f7aa407a8 Small Concern_spec cleanup 2018-01-29 15:13:04 +01:00
Pawel Chojnacki e3fe65543b Add Tests for Metrics::Concern 2018-01-29 15:13:04 +01:00
Pawel Chojnacki 0325c23732 Fix tests 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 91a991367e Convert active record 2018-01-29 15:13:03 +01:00
Pawel Chojnacki 66c1acba0b Fix code after refactoring 2018-01-29 15:13:03 +01:00
Pawel Chojnacki d97548daa8 cleanup method call 2018-01-29 15:13:03 +01:00
Douwe Maan 92e15071c1 Merge branch 'pawel/reduce_cardinality_of_prometheus_metrics' into 'master'
Reduce cardinality of some of GitLab's Prometheus metrics and fix observed duration reporting.

Closes #41045

See merge request gitlab-org/gitlab-ce!15881
2017-12-21 15:56:45 +00:00
Pawel Chojnacki 040167f072 Use seconds where possible, and convert to milliseconds for Influxdb consumption 2017-12-21 00:39:52 +01:00
Pawel Chojnacki 3c545133e8 Fix tests and formatting 2017-12-19 17:45:25 +01:00
Pawel Chojnacki db9e5bf75e fix test case description 2017-12-13 18:43:08 +01:00
Pawel Chojnacki da19ce625b Expire feature flag cache after 1minute 2017-12-12 18:52:08 +01:00
Pawel Chojnacki fd0a516854 use class variables instead of CONSTANTs 2017-12-12 18:38:28 +01:00
Pawel Chojnacki 408208bc2b Use AtomicFixNum to implement CAS isolated cache update.
i.e.
Using compare and swap we update the expires_at value.
The thread that actually is able to update this value will also set
the cache holding method_call enabled state
2017-12-12 18:38:28 +01:00
Pawel Chojnacki 5904b033db Implemente measurement enabled cache using AtomicReference 2017-12-12 18:38:28 +01:00
Pawel Chojnacki b503e6ff42 Implement simple in memory cache that expires after 5 minutes 2017-12-12 18:38:28 +01:00
Pawel Chojnacki 53dc9e83c3 Cache feature check for 5 minutes for MethodCall instrumentation toggle 2017-12-12 18:38:28 +01:00
Pawel Chojnacki a8ebed6016 Make `System.monotonic_time` retun seconds represented by float with microsecond precision 2017-12-12 18:12:49 +01:00
Stan Hu 10f5446c33 Remove RubySampler#sample_objects for performance as well 2017-12-11 06:55:59 -08:00
Stan Hu e1a8e5a509 Remove allocation tracking code from InfluxDB sampler for performance
On GitLab.com, InfluxSampler#sample_objects appears to take 1.2 s or so to
iterate through 1059 objects. This had led to delays of a couple hundred
milliseconds in processing in the main thread. Remove this code since it's not
really being used.

Closes gitlab-com/infrastructure#3250
2017-12-10 21:53:28 -08:00
Pawel Chojnacki 46cd2d93bb Use feature flag instead of application settigns to control if method calls should be instrumented 2017-11-23 23:39:10 +01:00
Pawel Chojnacki 0051b5fbcc Use only real duration to measure method call performance via Prometheus 2017-11-23 23:33:01 +01:00
Pawel Chojnacki efe4cab92b check method timing threshold when observing method performance 2017-11-23 23:33:01 +01:00
micael.bergeron 7fd3ce417f Revert "add metrics tagging to the sidekiq middleware"
This reverts commit d5859bb9d5.
This reverts commit 2b7e03cf69.
This reverts commit 7799a9bc44.
2017-11-08 16:21:08 -05:00
Pawel Chojnacki 90edcb45cd Make BackgroundTransaction#labels public
as are all other Transaction implementations
2017-11-07 19:23:52 +01:00
Pawel Chojnacki 765ddaeb85 Only enable unicorn metrics when not in sidekiq mode.
+ remove rebased test case
2017-11-02 21:27:28 +01:00
Pawel Chojnacki c9f0070b64 Make subscriber tests pass, after refactoring metrics 2017-11-02 18:18:16 +01:00
Pawel Chojnacki 735365a367 rename BackgroundMigration to SidekiqMigration 2017-11-02 18:18:16 +01:00
Pawel Chojnacki 30a4bb6628 Fix sidekiq middleware tests 2017-11-02 18:16:58 +01:00
Pawel Chojnacki 534f6b1125 Tests for Web transaction and remove simple transacton 2017-11-02 18:11:44 +01:00
Pawel Chojnacki 19634c259d Cleanup metrics names and removing unnecessary cache read total 2017-11-02 18:11:44 +01:00
Pawel Chojnacki c361c1e857 Adjust test to changed metrics methods.
+ remove deprecated test suite
2017-11-02 18:11:44 +01:00