Commit Graph

54 Commits

Author SHA1 Message Date
Stephanie Hingtgen ed408985fa
[release-12.0.1] K8s: Dashboards: Mark as v1beta1 (#104422)
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2025-04-23 19:21:59 +01:00
Ryan McKinley 0283c98e30
K8s/Folders: Use v1beta1 and app-sdk based spec (#103975) 2025-04-14 23:20:10 +03:00
Marco de Abreu c47ab101d1
Dashboards: Add Dashboard Schema validation (2) (#103844)
* Activate schema validation and align underlying systems

* update to save as v0 if not the right schema version

* Resolve merge conflicts

* Move RequireApiErrorStatus to tests package

* Add mutation tests

* Fix lint

* Only do min version check if dashboard is v1

* Fix lint and disable provisioning test

* Revert provisioning changes

* Revert more tests and add schema test

* Reran gen

* SQL Dashboard save

* Adjust APIVERSION

* Fixed mutation test

* Add logging on downgrade

---------

Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-04-11 23:05:41 +02:00
Ryan McKinley ac7edd3032
Provisioning: Skip validation when writing v1 dashbaords (#103893) 2025-04-11 16:04:00 +00:00
Roberto Jiménez Sánchez 4c21e7f8c2
Provisioning: small pending refactoring of pull request job (#103876)
* WIP: Refactor into evaluator and commenter

* Add mocks

* Fix existing tests comment

* Fix existing changes tests

* Use the extracted config

* Fix linting

* Remove trailing line
2025-04-11 15:18:41 +00:00
Ryan McKinley 664e5255fe
Provisioning: Use role based access when the target does not yet exist (#103862)
* role based fallback

* disable permissions cache with provisioning

* fallback to role based

* test with editor (not admin)

* test with editor (not admin)

* fix imports

* lint

* editor can create folders
2025-04-11 17:47:26 +03:00
Ryan McKinley 2c3422fc5c
Provisioning: return field paths in test error messages (#103850)
* Provisioning: Do not block connect step on error

* Display field errors

* Cleanup

* return field errors

* fix test

* convert errros to an array

* Fix history display

* Add getFormErrors

* metav1 issues

* lint

* Proper field names

* Fix notification

* Remove unused component

---------

Co-authored-by: Clarity-89 <homes89@ukr.net>
2025-04-11 14:26:42 +01:00
Stephanie Hingtgen f5ad1ef69b
K8s: Folders: Add v1 api (#103842) 2025-04-11 13:09:52 +01:00
Ryan McKinley 0d20680695
Provisioning: Improve PR feedback (#103537)
* fix ref

* dryRun before save

* reuse code for create vs update

* update comments

* can update PRs

* return useful errors

* update preview links

* lint fix

* update some wording

* update mocks

* fixed

* Update pkg/registry/apis/provisioning/resources/dualwriter.go

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* more updates

* fix link

* error string

* dry run

* refactor galore

* with template test

* very basic tests

* more test cases

* remove generator

* more tests

* fix lint

* multiple files now

* merge main

* merge main

* fix snapshots

* fix snapshots

* slugify title

---------

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
2025-04-11 11:06:16 +01:00
Roberto Jiménez Sánchez 2eaeff8ea7
Provisioning: Fix issue with double prefix for github repository (#103781) 2025-04-10 20:57:42 +03:00
Roberto Jiménez Sánchez 47664a7d51
Provisioning: unit test sync job (#103636)
* Split in multiple files

* Refactor sync even further

* Move more things between RepositoryResources

* Add status patcher

* Interface for sync functions

* Interface for compare function

* Add syncer back

* Move interfaces

* Move execute complete

* Return currentRef in syncer

* Add repository status test

* Add initial sync tests

* Fix a couple of spots

* Make initial sync tests work

* Fix register.go

* Add initial tests for sync worker

* Finish tests for sync

* Add incremental tests

* Add TODO

* Finish incremental tests

* Move folder creation to full sync

* Move interfaces

* Add initial full sync tests

* Update tests

* Reshape things

* Add changes test

* Fix register

* Add some tests

* Add more tests

* Add test

* WIP

* WIP: delete test

* Add more full test

* More tests

* Add tests for folder creation

* Add folder tests full sync

* Full coverage full sync

* Clean up tests

* Add more tests for changes function

* Enhance tests for Changes function to cover error scenarios and empty paths

- Added test cases for handling empty file paths and folder resources.
- Updated error message formatting in the Compare function for clarity.

* Add unit tests

* Failed initial patch

* Add tests failed repository resources

* Add test failed getting client

* Test for successful and unsuccessful syncs

* Add final tests for worker

* Fix existing tests

* Add missing test

* Fix spelling mistake

* Fix flake in changes test
2025-04-10 13:55:49 +01:00
Mariell Hoversholm 757be6365a
CI: Bump golangci-lint to 2.0.2 (#103572) 2025-04-10 14:42:23 +02:00
Ryan McKinley 236212aabb
Provisioning: ignore the grafanaAPIServerWithExperimentalAPIs flag (#103761) 2025-04-10 10:55:19 +03:00
Ryan McKinley 3f9e01b371
Provisioning: Use AccessChecker to verify if request has access to the parsed object (#103646) 2025-04-09 15:48:58 +01:00
Roberto Jiménez Sánchez 837f4864b1
Provisioning: unit test export job (#103620)
* Add repository resources interface for export worker

* Add mocks for repository resources

* Add unit tests for ExportWorker's IsSupported method

* Add unit tests for ExportWorker's Process method, covering scenarios for missing export settings, write permissions, branch restrictions, and client creation failures.

* Fix unit tests

* Single function

* Add more unit tests

* Add test for failed folder

* Fail export folder errors

* Add another test

* Positive folder export

* Too many folder export errors

* Too many errors on folder export

* Partial folder errors

* Add test for nested folder

* Add test dashboard export

* More cases

* Ignore existing dashboards

* Fix folder tests

* Fix clonable test

* Add clone failure test

* Add test clean up without push

* Working tests

* Use mock clonable

* Add unit tests for IsWriteAllowed

* Add behaviour to cover ref equal to configured branch

* Fix worker test

* Fix linting

* Split clone and push

* Wrapper for clone and push

* Separate methods for resources export

* Separate folder export

* Simplify single signature

* Refactor a bit more

* Separate folder export function

* Split it into different files

* Add FIXME

* Export function mock

* Export Resources tests

* Add test for cannot find client

* Check for branch

* Fix registry

* Move folder export tests

* Pass wrapper function

* Add worker tests

* Fail if branch is passed for clonable repositories

* Fix merge issues
2025-04-09 12:14:43 +02:00
Ryan McKinley 6df04f3949
Provisioning: Hold recent jobs in memory (#103678) 2025-04-09 12:46:01 +03:00
Ryan McKinley 410bb1cf74
Provisioning: Fix URL sanitization errors (#103640) 2025-04-09 10:33:55 +03:00
Ryan McKinley d3e6e308a0
K8s/ManagedBy: Enforce who can CRUD provisioning resources (#103322) 2025-04-08 14:17:33 +03:00
Roberto Jiménez Sánchez 4f05b42275
Provisioning: initial unit tests push job (#103577)
* Add repository resources interface for export worker

* Add mocks for repository resources

* Add unit tests for ExportWorker's IsSupported method

* Add unit tests for ExportWorker's Process method, covering scenarios for missing export settings, write permissions, branch restrictions, and client creation failures.

* Fix unit tests

* Single function

* Add more unit tests

* Add test for failed folder

* Fail export folder errors

* Add another test

* Positive folder export

* Too many folder export errors

* Too many errors on folder export

* Partial folder errors

* Add test for nested folder

* Add test dashboard export

* More cases

* Ignore existing dashboards

* Fix folder tests

* Fix clonable test

* Add clone failure test

* Add test clean up without push

* Working tests

* Use mock clonable

* Add unit tests for IsWriteAllowed

* Add behaviour to cover ref equal to configured branch

* Fix worker test

* Fix linting

* Split clone and push

* Wrapper for clone and push
2025-04-08 12:44:11 +02:00
Roberto Jiménez Sánchez fc099e9f0d
Provisioning: Run validation on config updates (#103265)
* Run validation on config updates

* Refactor code

* Add debug lines

* Remove test check on admission

* Organize imports

* Delegate events to the API client

* Extend error notification

* Deep copy default data

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Clarity-89 <homes89@ukr.net>
2025-04-07 14:48:28 +03:00
Ryan McKinley 1824694c72
Provisioning: Show branch in save form (#103513) 2025-04-07 12:48:32 +03:00
Ryan McKinley 1faa855ff1
Provisioning: Improve sample provisioning (#103508) 2025-04-07 12:37:18 +03:00
Ryan McKinley 8dbaeac9da
Provisioning: Use repository view rather than raw config (#103449) 2025-04-07 09:39:31 +03:00
Roberto Jiménez Sánchez 364247e26b
Provisioning: add most mocks to unit test provisioning jobs (#103446)
* Use interface for factories

* Add most mocks to unit test jobs

* Generate more mocks for main jobs package

* Interface for parser

* Use clients directly without parser

* Regenerate mock

* Eliminate resource clients from Parser

* Interface for folder tree
2025-04-04 15:42:20 +03:00
Roberto Jiménez Sánchez ea02e2e081
Provisioning: refactor dry-run and run logic to be stricter and more concise (#103357)
* Separate DryRun into separate method

* Fix linting

* Remove errors

* Remove checks in dualwriter

* Fix unit tests

* Add TODOs

* Dry Run as non-critical error

* Add TODOs

* Address TODO

* Fix tests

* Fix linting

* Deprecate dashboard name from path completely

* Use MissingName error also in parser

* Return 206 for non-critical errors

* Remove TODOs for previous dry-run
2025-04-04 13:31:48 +03:00
Ryan McKinley 04fb9f534e
Dashboards: Only expose LibraryPanels and search on v0 (not v1+v2) (#103335) 2025-04-03 20:24:12 +03:00
Ryan McKinley c5d76a8bba
Provisioning: Require a name in the saved resource (#103309) 2025-04-03 18:58:05 +03:00
Alex Khomenko edefc80c2a
Provisioning: Split active and finished jobs (#103351) 2025-04-03 17:38:12 +03:00
Roberto Jiménez Sánchez 800aa7827d
Provisioning: Ensure path to file exists in Files endpoints (#103343)
* Create ensure folder path exists

* Fix issue with folder names generated in 2 different ways
2025-04-03 13:50:41 +02:00
Ryan McKinley 5dad2b6c0f
Provisioning: Write finished job to history before delete (#103328) 2025-04-03 10:00:44 +03:00
Roberto Jiménez Sánchez aaa61d0919
Provisioning: Fix push to PR branch (#103262)
* Fix push to PR branch

* Fix original screenshot generation
2025-04-02 14:17:33 +03:00
Ryan McKinley 34192e3ad0
Provisioning: Support migrate to local (#103247) 2025-04-02 11:25:06 +03:00
Roberto Jiménez Sánchez 0aeefedb0c
Provisioning: Snapshot original dashboard (#103170)
Snapshot original dashboard
2025-04-02 10:12:42 +02:00
Roberto Jiménez Sánchez 04e152f326
Provisioning: webhook last event timestamp (#103180)
* Record webhook pinged event

* Add TODO for webhook creation updated

* Hack to wire client

* Revert accidental change in controller

* Wire the client

* Use factory method

* Remove omit empty

* Regenerate client

* Fix compilation

* Every 30 seconds if not pinged

* Move lines around

* Use different approach

* Added as part of the controller

* Exponential backoff for waiting for ping

* More stuff

* Revert changes in controller

* Add separate webhook section in overview

* Change order of translations

* Update ping within 1 minute

* Last event update

* Extract translation

* Display last event in frontend

* Refactor the logic around update

* Fix the type to marshal
2025-04-02 10:09:58 +02:00
Ryan McKinley f087d5f61b
Provisioning: validate files we read (#103197) 2025-04-01 17:49:08 +03:00
Roberto Jiménez Sánchez 29f395e1dd
Provisioning: introduce interface for git clones (#103175)
* Delegate clone to export in migrate from API server

* Clonable interface

* Root from register.go

* Call option push on write

* Fix linting
2025-04-01 13:42:52 +03:00
Roberto Jiménez Sánchez f91f739ee2
Remove unused HACK for testing new configuration (#103172) 2025-04-01 12:37:35 +02:00
Ryan McKinley ca35a89916
Provisioning: Manage repo jobs as single sub-resource (#103090) 2025-04-01 13:22:47 +03:00
Ryan McKinley 1ead18d856
Chore: Update git dependencies (#103159) 2025-04-01 13:10:50 +03:00
Roberto Jiménez Sánchez ccb127cc79
Remote provisioning: preview only one in pull requests (#103119)
* Limit number of previews to 10

* Use variable to check too many

* Single resource preview
2025-03-31 20:13:04 +02:00
Roberto Jiménez Sánchez 1ebcb3f604
Remote provisioning: fix webhook event processing (#103122)
Set the namespace in the context on Insert
2025-03-31 17:35:49 +02:00
Roberto Jiménez Sánchez 87aa2cba36
Remote provisioning: do not be that strict for history filepath (#103103)
Do not be that strict for history filepath
2025-03-31 16:50:19 +03:00
Roberto Jiménez Sánchez bb344fcd83
Remote provisioning: consolidate resource operations (#102972)
* Move to new repository

* Rename it to dual writer

* Rename the function

* Rename the methods

* Rename to exportResource

* Clean up logic in migrate and add TODOs

* Add TODOs

* Use generic client for unprovisioned

* ForEachResource

* More consolidation

* Refactor more around client

* Consolidate constants

* ForEachFolder

* More use of constants

* Add FIXME notes

* Use more constant

* Remove Dashboard

* Pass tree to folder manager

* Replicate tree

* Reduce export complexity

* More refactoring

* Use the ForEach for loading users

* Limit in-memory folders

* Isolate the object

* Improve the export function

* Move resources to resources package

* Move delete operation

* Move more logic

* More consolidation

* More renaming

* Fix more issues

* Ensure path exists when created a resource

* Simply append error

* Fix receiver lint issue

* Fix cyclomatic complexity

* Fix linting

* Remove folder path creation
2025-03-31 14:27:46 +02:00
Mariell Hoversholm 69803300f1
Provisioning: Complete rename of jobs (#102979)
Fixes: https://github.com/grafana/git-ui-sync-project/issues/164
2025-03-27 17:06:44 +01:00
Mariell Hoversholm 4cd1315700
App Platform(Provisioning): Listen to context cancel ASAP (#102957)
We should not write after the context is cancelled. This causes tests to fail as they are trying to clean up. To try to
accommodate this, we'll need to listen to the context in any loop where we don't immediately get other instructions from
e.g. trying to do an HTTP request.
2025-03-27 15:11:07 +01:00
Roberto Jiménez Sánchez b2d9b3abe6
Remote provisioning: use client discovery for sync job (#102885)
* Use client discovery for folders and dashboards synchronization

* Remove unused dashboards client

* Add folders import again
2025-03-27 11:07:06 +01:00
Roberto Jiménez Sánchez eff2da96d0
Remote Provisioning: Fix empty folder synchronization (#102789)
* Add behavior for hidden

* Rename to IsFilePathSupported

* Modify sync

* Revert renaming

* Fix the tests

* Add more tests

* Maintain empty folders until next full pulling

* Fix wording

* Consider the file as ignored

* Handle folder creation in sync

* Record folder creation / update

* Fix in manual test

* Ensure / slash for folders

* Refactor the tests

* Keep safe path

* Fix some cases

* Remove log lines
2025-03-26 13:39:22 +01:00
Stephanie Hingtgen 9e10a7f9ce
Tests: Fix failing integration test (#102847) 2025-03-25 21:17:56 -05:00
Mariell Hoversholm cc0c86b286
Provisioning: Fix migration: Go-git should pretend the Path is the repo root (#102755)
* fix: go git migration should pretend the path is the repo root

The migration was broken due to its having the path already in the repository, then adding it once more. This now
pretends the path is the repository root as intended.

* chore: remove test log

* fix: reduce permissions of directory

* chore: make update-workspace
2025-03-25 11:09:24 +01:00
Mariell Hoversholm 0536aa2d52
Provisioning: Gracefully handle AlreadyExists errors (#102758)
* Provisioning: Gracefully handle AlreadyExists errors

When this occurs, the job is already in the store. The vast majority of use-cases (which currently is all of them!) will
be fine with just accepting that as an OK.

* chore: make update-workspace

* fix: return error, but expect it in tests
2025-03-25 10:41:38 +01:00