diff --git a/.gitlab/issue_templates/Service Ping reporting and monitoring.md b/.gitlab/issue_templates/Service Ping reporting and monitoring.md new file mode 100644 index 00000000000..045f13c1c40 --- /dev/null +++ b/.gitlab/issue_templates/Service Ping reporting and monitoring.md @@ -0,0 +1,122 @@ + + +The [Product Intelligence group](https://about.gitlab.com/handbook/engineering/development/growth/product-intelligence/) runs manual reporting of ServicePing for GitLab.com on a weekly basis. This issue captures: + +- Captures the work required to complete the reporting process,. +- Captures the follow-up tasks that are focused on metrics performance verification. +- Identifies any potential issues. + +# New metrics to be verified + + + +# Failed metrics + +Broken metrics issues are marked with the ~"broken metric" label. + +# Use a detached screen session to generate Service Ping for GitLab.com + +## Prerequisites + +1. Make sure the SSH key is added to the local SSH agent: `ssh-add`. + +## Triggering + +1. Add the SSH key to the local SSH agent: `ssh-add`. +1. Connect to the bastion with SSH agent forwarding: `ssh -A lb-bastion.gprd.gitlab.com`. +1. Note which bastion host machine was assigned. For example: `@bastion-01-inf-gprd.c.gitlab-production.internal:~$` shows that you are connected to `bastion-01-inf-gprd.c.gitlab-production.internal`. +1. Create a named screen: `screen -S $USER-service-ping-$(date +%F)`. +1. Connect to the console host: `ssh $USER-rails@console-01-sv-gprd.c.gitlab-production.internal`. +1. Run: `ServicePing::SubmitService.new.execute`. +1. Press Control+a followed by Control+d to detach from the screen session. +1. Exit from the bastion: `exit`. + +## Verification (After approximately 30 hours) + +1. Reconnect to the bastion: `ssh -A lb-bastion.gprd.gitlab.com`. Make sure that you are connected to the same host machine that ServicePing was started on. For example, to connect directly to the host machine, use `ssh bastion-01-inf-gprd.c.gitlab-production.internal`. +1. Find your screen session: `screen -ls`. +1. Attach to your screen session: `screen -x 14226.mwawrzyniak_service_ping_2021_01_22`. +1. Check the last payload in the `raw_usage_data` table: `RawUsageData.last.payload`. +1. Check the when the payload was sent: `RawUsageData.last.sent_at`. + +## Stop the Service Ping process + +Use either of these processes: + +1. Reconnect to the bastion host machine. For example, use: `ssh bastion-01-inf-gprd.c.gitlab-production.internal`. +1. Find your screen session: `$ screen -ls`. +1. Attach to your screen session: `$ sudo -u screen -r`. +1. Press Control+c to stop the Service Ping process. + +OR + +1. Reconnect to the bastion host machine. For example, type: `ssh bastion-01-inf-gprd.c.gitlab-production.internal`. +1. List all process started by your username: `ps faux | grep `. +1. Locate the username that owns ServicePing reporting. +1. Send the kill signal for the ServicePing PID: `kill -9 `. + +## Service Ping process triggering (through a long-running SSH session) + +1. Connect to the `gprd` Rails console. +1. Run `SubmitUsagePingService.new.execute`. This process requires more than 30 hours to complete. +1. Find the last payload in the `raw_usage_data` table: `RawUsageData.last.payload`. +1. Check the when the payload was sent: `RawUsageData.last.sent_at`. + +```plaintext +ServicePing::SubmitService.new.execute + +# Get the payload +RawUsageData.last.payload + +# Time when payload was sent to VersionsAppp +RawUsageData.last.sent_at +``` + +# Verify Service Ping in VersionsApp + +To verify that the ServicePing was received in the VersionsApp do the following steps: + +1. Go to the VersionsApp console and locate: `RawUsageData.find(uuid: '')`. +1. Check the object. Either: + - Go to the Rails console and check the related `RawUsageData` object. + - Go to the VersionsApp UI . + +```ruby +/bin/herokuish procfile exec rails console + +puts UsageData.select(:recorded_at, :app_server_type).where(hostname: 'gitlab.com', uuid: 'ea8bf810-1d6f-4a6a-b4fd-93e8cbd8b57f').order('id desc').limit(5).to_json + +puts UsageData.find(21635202).raw_usage_data.payload.to_json +``` + +# Monitoring events tracked using Redis HLL + +Trigger some events from the User Interface. + +```ruby +Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'event_name', start_date: 28.days.ago, end_date: Date.current) +``` + +# What to do if you get mentioned + +In this issue, we keep the track of new metrics added to the Service Ping, and the metrics that are timing out. + +If you get mentioned, check the failing metric and open an optimization issue. + +# Service Ping manual generation for GitLab.com schedule + +| Generation start date | GitLab developer handle | Link to comment with payload | +| --------------------- | ----------------------- | ---------------------------- | +| 2022-04-18 | | | +| 2022-04-25 | | | +| 2022-05-02 | | | +| 2022-05-09 | | | +| 2022-05-16 | | | + + + +/confidential +/label ~"group::product intelligence" ~"devops::growth" ~backend ~"section::growth" ~"Category:Service Ping" +/epic https://gitlab.com/groups/gitlab-org/-/epics/6000 +/weight 5 +/title Monitor and Generate GitLab.com Service Ping diff --git a/Gemfile b/Gemfile index cc0277a547d..50ef5283522 100644 --- a/Gemfile +++ b/Gemfile @@ -100,7 +100,7 @@ gem 'grape-entity', '~> 0.10.0' gem 'rack-cors', '~> 1.0.6', require: 'rack/cors' # GraphQL API -gem 'graphql', '~> 1.11.10' +gem 'graphql', '~> 1.13.12' gem 'graphiql-rails', '~> 1.8' gem 'apollo_upload_server', '~> 2.1.0' gem 'graphql-docs', '~> 1.6.0', group: [:development, :test] diff --git a/Gemfile.lock b/Gemfile.lock index d95cf0d9d0b..41794b552ca 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -581,7 +581,7 @@ GEM faraday (>= 1.0) faraday_middleware graphql-client - graphql (1.11.10) + graphql (1.13.12) graphql-client (0.17.0) activesupport (>= 3.0) graphql (~> 1.10) @@ -1526,7 +1526,7 @@ DEPENDENCIES grape_logging (~> 1.8) graphiql-rails (~> 1.8) graphlient (~> 0.5.0) - graphql (~> 1.11.10) + graphql (~> 1.13.12) graphql-docs (~> 1.6.0) grpc (~> 1.42.0) gssapi diff --git a/app/assets/javascripts/error_tracking/components/error_details.vue b/app/assets/javascripts/error_tracking/components/error_details.vue index c369d879739..3fdf70e1621 100644 --- a/app/assets/javascripts/error_tracking/components/error_details.vue +++ b/app/assets/javascripts/error_tracking/components/error_details.vue @@ -301,7 +301,7 @@ export default { {{ __('More details') }} diff --git a/app/assets/javascripts/issuable/components/csv_export_modal.vue b/app/assets/javascripts/issuable/components/csv_export_modal.vue index b0af3612e05..736da92fa9f 100644 --- a/app/assets/javascripts/issuable/components/csv_export_modal.vue +++ b/app/assets/javascripts/issuable/components/csv_export_modal.vue @@ -1,16 +1,18 @@