* add db migration at service start time
* make changes for the 3 db
* revert migrator
* fix feature toggle check
Co-authored-by: Serge Zaitsev <serge.zaitsev@grafana.com>
* Remove URL-based alertmanagers from endpoint config
* WIP
* Add migration and alertmanagers from admin_configuration
* Empty comment removed
* set BasicAuth true when user is present in url
* Remove Alertmanagers from GET /admin_config payload
* Remove URL-based alertmanager configuration from UI
* Fix new uid generation in external alertmanagers migration
* Fix tests for URL-based external alertmanagers
* Fix API tests
* Add more tests, move migration code to separate file, and remove possible am duplicate urls
* Fix edge cases in migration
* Fix imports
* Remove useless fields and fix created_at/updated_at retrieval
Co-authored-by: George Robinson <george.robinson@grafana.com>
Co-authored-by: Konrad Lalik <konrad.lalik@grafana.com>
* clean up and document integration test convention
* clarify integration test conventions
* clean up integration tests that don't follow convention
* mark testIntegration* functions as helpers to avoid confusion
* add: added a repeat migration w. fixed checks for permissions
* add: migration to migrations
* refactor: fix migration instead of making a new one
* fix: removed the old id
* fix: keep old name but change id
* add: migration for patched previous migration
* add: migration from missing file
* Remove methods from sqlstore
* Remove commented out code
* Remove GetUserById from tests
* Adjust fake for get user profile
* Adjust test
* Adjust go mod files
* Try fix test
* Test adjustment
* Adjust test 2
* Remove commented out code
* feat: add new Folder table migration
Add a new folder table to support the Nested Folders feature.
https://github.com/grafana/grafana/issues/56880
* register nested folders feature flag (unused)
* feat: nested folder service (experiment)
This commit adds a NestedFolderSvc interface and stubbed out implementation as an alternative to the existing folder service. This is an experimental feature to try out different methods for backwards compatibility and parallelization, so that Grafana can continue to store folders in the existing (non-nested) manner while also using the new nested folder service.
Eventually the new service will (hopefully) become _the_ service, at which point the legacy service can be deprecated (or remain, with the new service methods replacing the original. whatever makes sense at the time).
* nested folders: don't run the new migration
This commit removes the nested folder migration from the list of active migrations so we can merge this branch and continue development without impacting Grafana instances built off main.
* Chore: Delete unused mocks
* Add back used methods
* Add back used mocks 2
* Add back used mocks 3
* Add back used mocks 4
* Add back the alerting mocks
* Remove mock variables
* Add bacl needed mock var
* chore: add alias for InitTestDB and Session
Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.
* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
* Alerting: Improve notification policies created during migration
Previously, migrated legacy alerts were connected to notification policies through
a `rule_uid` label in a 1:1 fashion. While this correctly mimicked pre-migration routing,
it didn't create a notification policy structure that is easy to view/modify. In addition,
having one policy per migrated alert is, in some ways, counter to the recommended approach of
Unified Alerting.
This change replaces `rule_uid`-based migrated notification policies with a private
label called `__contacts__`. This label stores a list of double quoted strings containing the names of
all contact points an AlertRule should route to (based on legacy notification channels). Finally,
one notification policy is created per contact point with each matching AlertRules via regex on this
`__contacts__` label.
The result is a simpler, clearer, and easier to modify notification policy structure, with the
added benefit that you can see which contact points an AlertRule is being routed to from the
AlertRule creation page.
* RBAC: Change the generate uid function to be deterministic so we can avoid collision
* RBAC: Use fmt.Errorf
* RBAC: Add comment
* RBAC: Export GenerateManagedRoleUID
Prior to this change, all alert instance writes and deletes happened
individually, in their own database transaction. This change batches up
writes or deletes for a given rule's evaluation loop into a single
transaction before applying it.
These new transactions are off by default, guarded by the feature toggle "alertingBigTransactions"
Before:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s
```
After:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s
```
So we cut time by about 75% and memory allocations by about 60% when
storing and deleting 100 instances.
* Copy sqlstore methods to suer store
* Adjust ProvideService signatures in test
* Add xorm tags and tests for search
* Remove methods from sqlstore
* fix lint in tests
* Chore: Copy user methods over to user store
* Fix some tests and bugs
* Add some more tests
* Move tests to user store
* Move back the tests
* Add some tests
* SQLStore: Ensure that sessions are always closed
Delete `NewSession()` in favour of `WithDbSession()`
* Add WithDbSessionForceNewSession to the interface
* Apply suggestions from code review
* add users-manager command
* add users-manager command
* rename files
* refactor: imports and renaming
* Command: add conflict merge user command
- MergeUser will
- replace all user_ids from conflicting users to the chosen userId
- delete users whose user_ids are not the chosen user
- SameIdentification will
- update chosen user with chosen email,login details
- delete users whose user_ids are not the chosen user
* refactor: clean up
* refactor: create structure for read, validate, ingest
* feat: ls and generate-file for conflicting users
* remove usagestats
* added back pkg/services/login/authinfoservice/database/stats.go
* Revert "added back pkg/services/login/authinfoservice/database/stats.go"
This reverts commit 2ba6e3c4d6.
* Revert "remove usagestats"
This reverts commit 1e3fa97810.
* cherry pick
* Revert "cherry pick"
This reverts commit 461626c306.
* validation of picked merge user
* fix test
* make lint
* make test run
* tests for ingest working
* clean up and refactored to align with downstream refactoring
* formatting
* refactor: name list instead of ls
* fix: static lint error use trimprefix
* WIP: permissions for validation
* fix: remove unused functions in sqlstore
* fix: remove unused function
* handling of multiple users and resolve discarded users
* fix tests
* fix: bug that did not exclude the blocks
* ioutil is blacklisted
* WIP: validation
* tests for merging a user working
* add latest changes to output print
* refactor: removed conflictEmail and conflictLogin that was not used
* refactor: code clean up, showChanges working
* test and linting fixes
* test and linting fixes
* refactor: removed logging of config and added more info for vlidation command
* refactor: fix order of code
* fix time now
* refactor: no longer need for check casesensitive login/email
* removed unnessecary loop
* refactor: move functions around
* test: working
* docs: add docuemntationf for file
* Add failing test for generating the conflict login block
* Fix regex
* Fix some stuff/tests
Co-authored-by: eleijonmarck <eric.leijonmarck@gmail.com>
* add: docs for conflict file
* add: conflict_email, conflict_login fields
* add: conflict_email, conflict_login fields
* WIP
* fix: tests working as intended
* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* review comments
* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go
Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go
Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
* missspelling
* trailing new line
* update to use userimpl store
* remove newline
* remove newline
* refactor: initializing of resolver for conflicts
* fix: test sqlStore
* refactor: removed lines
* refactor: remove TODOs
Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* Chore: Remove GetUserByEmail and GetUserByLogin from sqlstore
Rename GetUserProfile to GetProfile
* Fix lint
* Skip test for mysql
* Add missing method to sqlstore mock
* UserService: update callers to use the UserService instead of calling sqlstore directly
There is one major change hiding in this PR. UserService.Delete originally called a number of services to delete user-related records. I moved everything except the actual call to the user table, and moved those into the API. This was done to avoid dependencies cycles; many of our services depend on the user service, so the user service itself should have as few dependencies as possible.
Marcus Efraimsson
Sofia Papagiannaki
Ryan McKinley
ying-jeanne
Alex Moreno
Emil Tullstedt
Dan Cech
Eric Leijonmarck
Alexander Weaver
idafurjes
Kristin Laemmert
Karl Persson
owensmallwood
Matthew Jacobson
Yuriy Tseretyan
Joe Blubaugh
Piotr Jamróz
Jean-Philippe Quéméner