Commit Graph

53 Commits

Author SHA1 Message Date
Peter Štibraný c4d3eb1cd0
Remove support for Google Spanner database. (#105846)
* Remove support for Google Spanner database.
2025-05-23 11:35:59 +02:00
Serge Zaitsev 694b9dfe50
Chore: Replace xorm.io/xorm imports (#104458)
* replace xorm.io/xorm imports

* replace xorm from other go.mod files

* clean up workspace

* nolint does not make sense anymore as it is not a module

* try if nolint directive helps

* use nolint:all for xorm

* add more nolints

* try to skip xorm in linter config

* exclude xorm differently

* retrigger ci
2025-05-02 17:13:01 +02:00
Peter Štibraný f7f28757f6
Skip noop migrations, instead of executing SELECT 0. (#103086)
* Skip noop migrations, instead of executing SELECT 0.
2025-03-31 15:19:32 +02:00
Peter Štibraný 0ce67dad74
Spanner: support for lastInsertId (#101770)
* Support for returning inserted ID in Spanner.
* Convert bytes to string when storing them into text column.
2025-03-10 15:59:06 +01:00
Peter Štibraný 9e690bd620
Spanner support enhancements (#101634)
* Adds ability to run integration tests against spanner (by using GRAFANA_TEST_DB=spanner env variable. SPANNER_DB variable then specifies database to use: spannertest, emulator or string like /projects/<project>/instances/<instance>/databases/<db>)
* Adds feature to migration dialects to create database from a snapshot, instead of running individual migrations.
* Adds first version of Spanner snapshot, prepared from "OSS" migrations.
* Uses generated bit-reversed-positive values instead of auto_increment. (As an experiment)
2025-03-06 16:11:20 +01:00
Serge Zaitsev 165bca6417
Chore: Add initial/experimental xorm spanner driver (#101398)
* make it build and start

* run some migrations

* add build tags, remove log

* remove unused code

* revert go.mod changes

* move initialisation into dialect file

* update workspace

* update workspace once again

* clean up dependencies

* further cleanup

* Address some review feedback.

* Fix go.sum.

---------

Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
2025-03-03 17:02:10 +01:00
Dave Henderson ced7a84f8b
chore(metrics): Add metrics & traces to DB migration (#97181)
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-12-05 17:22:19 -05:00
Dan Cech 5dfe4cf407
Storage: Watch tests (#85496)
* basic watch tests working

* refactor to read previous event within poller

* add watch test files

* cleanup

* watch tests passing

* debug cleanup

* special handling for canceled context

* cleanup

* fix wire

* fix sqlite_sequence cleanup issue

* move watch tests to integration, wait for provisioned dashboards

* handle context deadline exceeded and eof errors

* add comment about sleep
2024-04-08 11:42:12 -04:00
Dan Cech 1f1461734c
Storage: Add support for sortBy selector (#80680)
* add support for sortBy field selector

* use label selectors instead of field selectors

* set entity_labels on create & update

* make entity server integration tests work

* test fixes

* be more consistent with handling of empty body, meta or status

* workaround for database is locked errors during migration

* fix double import of sqlite3

* rename functions and tidy up

* refactor update

* disable integration tests until we can fix the database locking issue
2024-02-07 15:05:10 -05:00
Dan Cech 67b2972052
Chore: add/update sqlstore-related helper functions (#77408)
* add/update sqlstore-related helper functions

* add documentation & tests for InsertQuery and UpdateQuery, make generated SQL deterministic by sorting columns

* remove old log line
2023-11-03 10:30:52 -04:00
Marcus Efraimsson edc34fc836
Instrumentation: Improve instrumentation for database migrations (#74659) 2023-09-11 14:10:21 +03:00
Dan Cech a6279b2d62
Database: Make dialects independent of xorm Engine (#69955)
* make dialects independent of xorm Engine

* goimports
2023-06-14 16:13:36 -04:00
Ryan McKinley 26658d172f
Migrations: Support independent migration log for entity api (#68511) 2023-05-22 11:31:07 -07:00
Ieva 035bf29146
RBAC: Remove the option to disable RBAC and add automated permission migrations for instances that had RBAC disabled (#66652)
* RBAC: Stop reading enabeld from ini file and always set to true

* Migrations: Add a migration for rbac to reset data migrations if rbac
was disabled

* If rbac was disabled we reset the data and data migrations that rbac
  has to perform to get it to a correct state

* Migrator: Store migration logs on migrator and add function to clear it from the
in-memory stored logs

* update tests

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-04-19 16:34:19 +01:00
Kyle Brandt 55d2d872ec
Chore: Missed deprecations due to overly broad lint exclusion (#59732) 2022-12-14 12:32:45 +01:00
Kat Yang 31630edf0c
Chore: Remove Wrapf (#50128)
* Chore: Remove Wrapf

* Remove all Wrapf refs

* Remove last Wrapf ref

* Fix lint errors

* Remove Wrap and Wrapf definitions

* Remove unnecessary colon
2022-06-06 22:30:31 +02:00
Joan López de la Franca Beltran 219e848e73
SQLStore: Prevent migration_id duplicates (#47774)
* SQLStore: Prevent migration_id duplicates

* Migrations: Remove non-executed migration (duplicated id)

Co-authored-by: Leonard Gram <leo@xlson.com>
2022-06-04 01:59:49 +02:00
Kat Yang 3c3039f5b3
Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
Sofia Papagiannaki d718ee1918
SQLStore: Prevent concurrent migrations (#44101)
* SQLStore: Prevent concurrent migrations

* Hide behind a feature toggle

* Configurable locking attempt timeout

* Update docs/sources/administration/configuration.md

Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2022-02-15 18:54:27 +02:00
Armand Grillet 6523486122
Alerting: Make Unified Alerting enabled by default for those who do not use legacy alerting (#42200)
* update AlertingEnabled and UnifiedAlertingSettings.Enabled to be pointers
* add a pseudo migration to fix the AlertingEnabled and UnifiedAlertingSettings.Enabled if the latter is not defined
* update the default configuration file to make default value for both 'enabled' flags be undefined

Misc
* update Migrator to expose DB engine. This is needed for a ualert migration to access the database while the list of migrations is created.
* add more verbose failure when migrations do not match

Co-authored-by: gotjosh <josue@grafana.com>
Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
2021-11-24 14:56:07 -05:00
Sofia Papagiannaki 04d5dcb7c8
Alerting: modify DB table, accessors and migration to restrict org access (#37414)
* Alerting: modify table and accessors to limit org access appropriately

* Update migration to create multiple Alertmanager configs

* Apply suggestions from code review

Co-authored-by: gotjosh <josue@grafana.com>

* replace mg.ClearMigrationEntry()

mg.ClearMigrationEntry() would create a new session.
This commit introduces a new migration for clearing an entry from migration log for replacing  mg.ClearMigrationEntry() so that all dashboard alert migration operations will run inside the same transaction.
It adds also `SkipMigrationLog()` in Migrator interface for skipping adding an entry in the migration_log.

Co-authored-by: gotjosh <josue@grafana.com>
2021-08-12 16:04:09 +03:00
Sofia Papagiannaki 423474325a
Alerting: fix postgres migration on conflict (#35556) 2021-06-14 16:43:33 +03:00
Kyle Brandt dc3e17ba5b
Alerting: Change rule migration to be based on feature flag (#33792)
makes it so the feature flag can be turned on off, and the migration will be cleared and rerun. All existing NG alert rules, configuration settings, etc are removed when disabling the feature flag.
for https://github.com/grafana/alerting-squad/issues/142

Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
2021-05-11 08:08:39 -04:00
Alexander Emelin 0c2bcbf2bc
Live: persisting last message in cache for broadcast scope (#32938) 2021-04-30 11:06:33 -07:00
Carl Bergquist c899bf3592
log skipped, performed and duration for migrations (#31722)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2021-03-07 21:26:29 +01:00
Arve Knudsen 25048ebdf8
Chore: Add CloudWatch HTTP API tests (#29691)
* CloudWatch: Add HTTP API tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-01-07 11:36:13 +01:00
Sofia Papagiannaki 3cac10e598
AlertingNG: Create a scheduler to evaluate alert definitions (#29305)
* Always use cache: stop passing skipCache among ngalert functions

* Add updated column

* Scheduler initial draft

* Add retry on failure

* Allow settting/updating alert definition interval

Set default interval if no interval is provided during alert definition creation.
Keep existing alert definition interval if no interval is provided during alert definition update.

* Parameterise alerting.Ticker to run on custom interval

* Allow updating alert definition interval without having to provide the queries and expressions

* Add schedule tests

* Use xorm tags for having initialisms with consistent case in Go

* Add ability to pause/unpause the scheduler

* Add alert definition versioning

* Optimise scheduler to fetch alert definition only when it's necessary

* Change MySQL data column to mediumtext

* Delete alert definition versions

* Increase default scheduler interval to 10 seconds

* Fix setting OrgID on updates

* Add validation for alert definition name length

* Recreate tables
2020-12-17 16:00:09 +02:00
Arve Knudsen 9593d57914
Chore: Enable errorlint linter (#29227)
* Enable errorlint linter
* Handle wrapped errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-11-19 14:47:17 +01:00
Arve Knudsen b5379c5335
Chore: Fix SQL related Go variable naming (#28887)
* Chore: Fix variable naming

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-11 06:21:08 +01:00
Arve Knudsen 102448040d
SQLite: Set 0640 permissions on SQLite database file (#26339)
* SQLite: Set 640 permissions on SQLite database file
2020-07-23 15:47:26 +02:00
Arve Knudsen d1e6214a4a
Chore: Enable scopelint Go linter (#25896)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 17:04:38 +02:00
Noval Agung Prayogo ea35ae4d1f
Chore: Update the xorm dependency (#22376)
* Upgrade xorm

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-04-01 15:57:21 +02:00
Arve Knudsen 2a78d2a61c
pkg/services: Check errors (#19712)
* pkg/services: Check errors
* pkg/services: Don't treat context.Canceled|context.DeadlineExceeded as error
2019-10-22 14:08:18 +02:00
zhulongcheng 2fff8f77dc move log package to /infra (#17023)
ref #14679

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-05-13 08:45:54 +02:00
Marcus Efraimsson 12f3c8f9d6
fix only add column if not exists for mysql 2018-12-18 23:02:08 +01:00
Marcus Efraimsson 09b3014683
fix only create/drop database indices if not exists/exists 2018-12-18 21:47:45 +01:00
Torkel Ödegaard 92ed1f04af sql: added code migration type 2018-08-21 13:53:04 +02:00
Torkel Ödegaard 6c0752473a
Sqlstore refactor (#11908)
* refactor: tracing service refactoring

* refactor: sqlstore to instance service

* refactor: sqlstore & registory priority

* refactor: sqlstore refactor wip

* sqlstore: progress on getting tests to work again

* sqlstore: progress on refactoring and getting tests working

* sqlstore: connection string fix

* fix: not sure why this test is not working and required changing expires

* fix: updated grafana-cli
2018-05-18 11:10:10 +02:00
Dan Cech 1dfff74da9 move database-specific code into dialects (#11884) 2018-05-10 16:54:21 +02:00
Karsten Weiss de8696d5d3 Outdent code after if block that ends with return (golint)
This commit fixes the following golint warnings:
pkg/bus/bus.go:64:9: if block ends with a return statement, so drop this else and outdent its block
pkg/bus/bus.go:84:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:137:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:177:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:183:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:199:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:208:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/components/dynmap/dynmap.go:236:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:242:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:257:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:263:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:278:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:284:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:299:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:331:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:350:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:356:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:366:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:390:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:396:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:405:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:427:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:433:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:442:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:459:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:465:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:474:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:491:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:497:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:506:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:523:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:529:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:538:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:555:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:561:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:570:12: if block ends with a return statement, so drop this else and outdent its block
pkg/login/ldap.go:55:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/login/ldap_test.go:372:10: if block ends with a return statement, so drop this else and outdent its block
pkg/middleware/middleware_test.go:213:12: if block ends with a return statement, so drop this else and outdent its block
pkg/plugins/dashboard_importer.go:153:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/dashboards_updater.go:39:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/dashboards_updater.go:121:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/plugins.go:210:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/plugins.go:235:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/eval_context.go:111:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifier.go:92:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifier.go:98:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifier.go:122:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/rule.go:108:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/rule.go:118:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/rule.go:121:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifiers/telegram.go:94:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/annotation.go:34:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/sqlstore/annotation.go:99:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/sqlstore/dashboard_test.go:107:13: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/plugin_setting.go:78:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/preferences.go:91:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/user.go:50:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/migrator/migrator.go:106:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/sqlstore/migrator/postgres_dialect.go:48:10: if block ends with a return statement, so drop this else and outdent its block
pkg/tsdb/time_range.go:59:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/tsdb/time_range.go:67:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/tsdb/cloudwatch/metric_find_query.go:225:9: if block ends with a return statement, so drop this else and outdent its block
pkg/util/filepath.go:68:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
2018-04-27 23:10:43 +02:00
Daniel Lee 1c20126f87 database: update xorm to v0.6.4 and xorm core to v0.5.7 2018-03-16 00:02:50 +01:00
bergquist 56907eef69 tests: makes sure we all migrations are working 2018-02-14 15:30:29 +01:00
Torkel Ödegaard 5ec86a9ef4 fix(build): fixed broken build, unreachable code in migrator.go 2016-09-30 09:23:26 +02:00
Torkel Ödegaard 2b8177e3e5 fix(migration): fix for sqlstore migration, the execution of the migration and recording of the success of it was not done in same transaction, fixes #5315 2016-09-30 07:49:40 +02:00
Torkel Ödegaard 086b59483e feat(loggin): disable logging by default so unit tests are not full of logging 2016-06-11 11:38:25 +02:00
Torkel Ödegaard a02cf5beb7 feat(logging): added loging filters 2016-06-07 12:11:41 +02:00
Torkel Ödegaard 22778e6efd feat(logging): a lot of progress on moving to new logging lib, #4590 2016-06-06 23:06:44 +02:00
Torkel Ödegaard 16fa5c4df3 fix(mysql): fix for migration in newly added temp_user table, fixes #2509 2015-08-14 09:41:07 +02:00
Torkel Ödegaard 563d5e3ad8 All migrations work in sqlite3, mysql and postgres 2015-02-25 07:57:51 +01:00
Torkel Ödegaard 02a89c752b Progress on database schema migration for account -> org refactor 2015-02-24 11:46:34 +01:00