grafana/pkg/build/e2e-playwright/e2e.go

115 lines
2.9 KiB
Go
Raw Permalink Normal View History

E2E: migrate to playwright (#107241) * separate playwright into its own folder * better separation * add login scenario, add tags * remove ui option * update CODEOWNERS * add a panels suite test * migrate queries test to playwright * rename + add dashlist test * add panelEdit_base * add geomap map controls test * add geomap-layer-types * add geomap-spatial-operations tests * add frontend-sandbox-panel tests * add smoke-tests-suite * add comment about adding datasource * add dashboard-browse-nested * add dashboard-browse * add dashboard-export-json * add dashboard-keybindings test * remove @wip tag * turn on screenshots and add comment for why this test fails * add dashboard-links-without-slug test * try adding permissions in the test as well * add dashboard-live-streaming * context in the test doesn't work - sad * create dashboard-public-templating * add dashboard-public-create and make live streaming more resilient * add share externally test * add dashboard-share-internally * add share-snapshot-create test * add dashboard-templating * add timepicker tests * add embedded-dashboard test * add general_dashboards test * add import-dashboard test * add load-options-from-url test * add new-constant-variable test * add custom-variable test * add new-datasource-variable test * add new-interval-variable test * add text-box-variable test * add new-query-variable test * add horizontal repeat test * add panel-vertical-repeat test * add empty-row-repeat test * add set-options-from-ui test * add snapshot-create test * add templating test * add textbox-variables test * add cloud-plugins-suite * add storybook verification tests * add playwright storybook verification workflow * add playwright browsers * update CODEOWNERS * test change to trigger storybook verification workflows * try container instead * get the version right... * go back to installing - less chance of forgetting to update * Basic Github Actions Squashed commit of the following: commit f84c650a71bba339da7f96303292717569d4caa6 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 13:23:46 2025 +0100 add arg for sharding, but not using it yet commit 7bcf0512c6652e079757e7f892d1802eb255a6a7 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:30:30 2025 +0100 less newline commit b6439118829a09a4c8c49d7692d38df0dff7e7ae Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:24:31 2025 +0100 less logs commit 38f871e9c235344f12ef31121fdfced549c5323c Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 10:00:26 2025 +0100 fix yaml commit db9a773136eff40a82beb3b2f55f73ab0d07a717 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:57:47 2025 +0100 clean up files commit c0525f41fa8b498f2fdf71b5047301b0e2945498 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:44:56 2025 +0100 gha workflow commit 895bea7c527bfbb148a421b9283fa0805f6ab11d Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 19:33:08 2025 +0100 working dagger commit cea1f8443738e9e4b375c065dd4a1e466036bb74 Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 16:17:46 2025 +0100 wip * shard gha * some tidy up * add flags for exporting results, and a gha step to merge runs * fix shard gha * add dashboard-duplicate-panel test * add dashboard-outline test * add dashboards-add-panel * remove some commented out code * add dashboards-title-description test * add dashboards-remove-panel * don't install cypress * gha: check playwright results * add dashboards-edit-adhoc-variables test * fix check-jobs * add dagger cloud token * add dagger cloud token * add edit-datasource-variable test * update CODEOWNERS * add dashboards-edit-group-by-variables (skipped for now) * add dashboards-edit-panel-title-description test * add dashboards-edit-transparent-bg test * add dashboards-edit-query-variables test * run with 8 shards * add dashboards-edit-variables * tidy up gha * add dashboard-group-panels * fix action * try to cache the grafana build * fix missing action becuase no checkout, use builtin continue-on-error instead * fix missing id * cat out.txt * debug build cache * fix debug build cache * add dashboards-panel-layouts test * tidy up * no more debug * fix grafana dir * add dashboards-move-panel test * skip some failing tests * mark up plugins tests with @plugins tag, only run @plugins tests in drone * Hackathon/Playwright Conversion - Various Suite (#107516) * Playwright Migration: Various Suite tests * skipping bad tests * fix some tests that can fail * fix uid * separate user for the verify-i18n test * build test plugins for grafana server * properly blur input fields * login manually * get dashboardPage from goto * ignore a couple of type assertions * remove a couple of timeouts * remove timeouts on dashboard-share-internally * use toBeHidden * make dashboard-share-internally more stable * remove TEMP_DAGGER_TOKEN * clean up visaulization-suggestions * unskip gauge test * unskip trace-view-scrolling * attempt to make create variable utils stable * unskip loki tests * make go linter happy * unskip edit-group-by-variables test * unskip move panel tests * isolate dashboard-timepicker tests with separate user * create data source as part of smoke test * make sure we're awaiting in dashboard-edit-adhoc-variables * make dashboards-edit-variables test more robust * Hackathon Playwright: Dashboards Search (#107580) * Hackathon Playwright: Dashboards Search * Feedback changes * make trace-view-scrolling more stable * add json report and bench step * fix bench version * move fail step to after the playwright report so we can report test failures * fix output file name * fix typo * try wrap in expect.poll * stability * bit more tidy up * fix dashboard-new-layouts tests * move test-plugins to e2e-playwright * fix go code for drone e2e run * move loki plugin-e2e test * make v2 dashboards work again --------- Co-authored-by: joshhunt <josh.hunt@grafana.com> Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com> Co-authored-by: Collin Fingar <collin.fingar@grafana.com> Co-authored-by: Jeff Levin <jeff@levinology.com>
2025-07-11 17:31:33 +08:00
package main
import (
"context"
"fmt"
"strings"
"dagger.io/dagger"
)
var (
// Locations in the container to write results to
testResultsDir = "/playwright-test-results"
htmlResultsDir = "/playwright-html-report"
blobResultsDir = "/playwright-blob-report"
)
type RunTestOpts struct {
GrafanaService *dagger.Service
FrontendContainer *dagger.Container
HostSrc *dagger.Directory
Shard string
HTMLReportExportDir string
BlobReportExportDir string
TestResultsExportDir string
PlaywrightCommand string
CloudPluginCreds *dagger.File
E2E: migrate to playwright (#107241) * separate playwright into its own folder * better separation * add login scenario, add tags * remove ui option * update CODEOWNERS * add a panels suite test * migrate queries test to playwright * rename + add dashlist test * add panelEdit_base * add geomap map controls test * add geomap-layer-types * add geomap-spatial-operations tests * add frontend-sandbox-panel tests * add smoke-tests-suite * add comment about adding datasource * add dashboard-browse-nested * add dashboard-browse * add dashboard-export-json * add dashboard-keybindings test * remove @wip tag * turn on screenshots and add comment for why this test fails * add dashboard-links-without-slug test * try adding permissions in the test as well * add dashboard-live-streaming * context in the test doesn't work - sad * create dashboard-public-templating * add dashboard-public-create and make live streaming more resilient * add share externally test * add dashboard-share-internally * add share-snapshot-create test * add dashboard-templating * add timepicker tests * add embedded-dashboard test * add general_dashboards test * add import-dashboard test * add load-options-from-url test * add new-constant-variable test * add custom-variable test * add new-datasource-variable test * add new-interval-variable test * add text-box-variable test * add new-query-variable test * add horizontal repeat test * add panel-vertical-repeat test * add empty-row-repeat test * add set-options-from-ui test * add snapshot-create test * add templating test * add textbox-variables test * add cloud-plugins-suite * add storybook verification tests * add playwright storybook verification workflow * add playwright browsers * update CODEOWNERS * test change to trigger storybook verification workflows * try container instead * get the version right... * go back to installing - less chance of forgetting to update * Basic Github Actions Squashed commit of the following: commit f84c650a71bba339da7f96303292717569d4caa6 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 13:23:46 2025 +0100 add arg for sharding, but not using it yet commit 7bcf0512c6652e079757e7f892d1802eb255a6a7 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:30:30 2025 +0100 less newline commit b6439118829a09a4c8c49d7692d38df0dff7e7ae Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:24:31 2025 +0100 less logs commit 38f871e9c235344f12ef31121fdfced549c5323c Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 10:00:26 2025 +0100 fix yaml commit db9a773136eff40a82beb3b2f55f73ab0d07a717 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:57:47 2025 +0100 clean up files commit c0525f41fa8b498f2fdf71b5047301b0e2945498 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:44:56 2025 +0100 gha workflow commit 895bea7c527bfbb148a421b9283fa0805f6ab11d Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 19:33:08 2025 +0100 working dagger commit cea1f8443738e9e4b375c065dd4a1e466036bb74 Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 16:17:46 2025 +0100 wip * shard gha * some tidy up * add flags for exporting results, and a gha step to merge runs * fix shard gha * add dashboard-duplicate-panel test * add dashboard-outline test * add dashboards-add-panel * remove some commented out code * add dashboards-title-description test * add dashboards-remove-panel * don't install cypress * gha: check playwright results * add dashboards-edit-adhoc-variables test * fix check-jobs * add dagger cloud token * add dagger cloud token * add edit-datasource-variable test * update CODEOWNERS * add dashboards-edit-group-by-variables (skipped for now) * add dashboards-edit-panel-title-description test * add dashboards-edit-transparent-bg test * add dashboards-edit-query-variables test * run with 8 shards * add dashboards-edit-variables * tidy up gha * add dashboard-group-panels * fix action * try to cache the grafana build * fix missing action becuase no checkout, use builtin continue-on-error instead * fix missing id * cat out.txt * debug build cache * fix debug build cache * add dashboards-panel-layouts test * tidy up * no more debug * fix grafana dir * add dashboards-move-panel test * skip some failing tests * mark up plugins tests with @plugins tag, only run @plugins tests in drone * Hackathon/Playwright Conversion - Various Suite (#107516) * Playwright Migration: Various Suite tests * skipping bad tests * fix some tests that can fail * fix uid * separate user for the verify-i18n test * build test plugins for grafana server * properly blur input fields * login manually * get dashboardPage from goto * ignore a couple of type assertions * remove a couple of timeouts * remove timeouts on dashboard-share-internally * use toBeHidden * make dashboard-share-internally more stable * remove TEMP_DAGGER_TOKEN * clean up visaulization-suggestions * unskip gauge test * unskip trace-view-scrolling * attempt to make create variable utils stable * unskip loki tests * make go linter happy * unskip edit-group-by-variables test * unskip move panel tests * isolate dashboard-timepicker tests with separate user * create data source as part of smoke test * make sure we're awaiting in dashboard-edit-adhoc-variables * make dashboards-edit-variables test more robust * Hackathon Playwright: Dashboards Search (#107580) * Hackathon Playwright: Dashboards Search * Feedback changes * make trace-view-scrolling more stable * add json report and bench step * fix bench version * move fail step to after the playwright report so we can report test failures * fix output file name * fix typo * try wrap in expect.poll * stability * bit more tidy up * fix dashboard-new-layouts tests * move test-plugins to e2e-playwright * fix go code for drone e2e run * move loki plugin-e2e test * make v2 dashboards work again --------- Co-authored-by: joshhunt <josh.hunt@grafana.com> Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com> Co-authored-by: Collin Fingar <collin.fingar@grafana.com> Co-authored-by: Jeff Levin <jeff@levinology.com>
2025-07-11 17:31:33 +08:00
}
func RunTest(
ctx context.Context,
d *dagger.Client,
opts RunTestOpts,
) (*dagger.Container, error) {
playwrightCommand := buildPlaywrightCommand(opts)
grafanaHost, err := opts.GrafanaService.Hostname(ctx)
if err != nil {
return nil, err
}
E2E: migrate to playwright (#107241) * separate playwright into its own folder * better separation * add login scenario, add tags * remove ui option * update CODEOWNERS * add a panels suite test * migrate queries test to playwright * rename + add dashlist test * add panelEdit_base * add geomap map controls test * add geomap-layer-types * add geomap-spatial-operations tests * add frontend-sandbox-panel tests * add smoke-tests-suite * add comment about adding datasource * add dashboard-browse-nested * add dashboard-browse * add dashboard-export-json * add dashboard-keybindings test * remove @wip tag * turn on screenshots and add comment for why this test fails * add dashboard-links-without-slug test * try adding permissions in the test as well * add dashboard-live-streaming * context in the test doesn't work - sad * create dashboard-public-templating * add dashboard-public-create and make live streaming more resilient * add share externally test * add dashboard-share-internally * add share-snapshot-create test * add dashboard-templating * add timepicker tests * add embedded-dashboard test * add general_dashboards test * add import-dashboard test * add load-options-from-url test * add new-constant-variable test * add custom-variable test * add new-datasource-variable test * add new-interval-variable test * add text-box-variable test * add new-query-variable test * add horizontal repeat test * add panel-vertical-repeat test * add empty-row-repeat test * add set-options-from-ui test * add snapshot-create test * add templating test * add textbox-variables test * add cloud-plugins-suite * add storybook verification tests * add playwright storybook verification workflow * add playwright browsers * update CODEOWNERS * test change to trigger storybook verification workflows * try container instead * get the version right... * go back to installing - less chance of forgetting to update * Basic Github Actions Squashed commit of the following: commit f84c650a71bba339da7f96303292717569d4caa6 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 13:23:46 2025 +0100 add arg for sharding, but not using it yet commit 7bcf0512c6652e079757e7f892d1802eb255a6a7 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:30:30 2025 +0100 less newline commit b6439118829a09a4c8c49d7692d38df0dff7e7ae Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:24:31 2025 +0100 less logs commit 38f871e9c235344f12ef31121fdfced549c5323c Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 10:00:26 2025 +0100 fix yaml commit db9a773136eff40a82beb3b2f55f73ab0d07a717 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:57:47 2025 +0100 clean up files commit c0525f41fa8b498f2fdf71b5047301b0e2945498 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:44:56 2025 +0100 gha workflow commit 895bea7c527bfbb148a421b9283fa0805f6ab11d Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 19:33:08 2025 +0100 working dagger commit cea1f8443738e9e4b375c065dd4a1e466036bb74 Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 16:17:46 2025 +0100 wip * shard gha * some tidy up * add flags for exporting results, and a gha step to merge runs * fix shard gha * add dashboard-duplicate-panel test * add dashboard-outline test * add dashboards-add-panel * remove some commented out code * add dashboards-title-description test * add dashboards-remove-panel * don't install cypress * gha: check playwright results * add dashboards-edit-adhoc-variables test * fix check-jobs * add dagger cloud token * add dagger cloud token * add edit-datasource-variable test * update CODEOWNERS * add dashboards-edit-group-by-variables (skipped for now) * add dashboards-edit-panel-title-description test * add dashboards-edit-transparent-bg test * add dashboards-edit-query-variables test * run with 8 shards * add dashboards-edit-variables * tidy up gha * add dashboard-group-panels * fix action * try to cache the grafana build * fix missing action becuase no checkout, use builtin continue-on-error instead * fix missing id * cat out.txt * debug build cache * fix debug build cache * add dashboards-panel-layouts test * tidy up * no more debug * fix grafana dir * add dashboards-move-panel test * skip some failing tests * mark up plugins tests with @plugins tag, only run @plugins tests in drone * Hackathon/Playwright Conversion - Various Suite (#107516) * Playwright Migration: Various Suite tests * skipping bad tests * fix some tests that can fail * fix uid * separate user for the verify-i18n test * build test plugins for grafana server * properly blur input fields * login manually * get dashboardPage from goto * ignore a couple of type assertions * remove a couple of timeouts * remove timeouts on dashboard-share-internally * use toBeHidden * make dashboard-share-internally more stable * remove TEMP_DAGGER_TOKEN * clean up visaulization-suggestions * unskip gauge test * unskip trace-view-scrolling * attempt to make create variable utils stable * unskip loki tests * make go linter happy * unskip edit-group-by-variables test * unskip move panel tests * isolate dashboard-timepicker tests with separate user * create data source as part of smoke test * make sure we're awaiting in dashboard-edit-adhoc-variables * make dashboards-edit-variables test more robust * Hackathon Playwright: Dashboards Search (#107580) * Hackathon Playwright: Dashboards Search * Feedback changes * make trace-view-scrolling more stable * add json report and bench step * fix bench version * move fail step to after the playwright report so we can report test failures * fix output file name * fix typo * try wrap in expect.poll * stability * bit more tidy up * fix dashboard-new-layouts tests * move test-plugins to e2e-playwright * fix go code for drone e2e run * move loki plugin-e2e test * make v2 dashboards work again --------- Co-authored-by: joshhunt <josh.hunt@grafana.com> Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com> Co-authored-by: Collin Fingar <collin.fingar@grafana.com> Co-authored-by: Jeff Levin <jeff@levinology.com>
2025-07-11 17:31:33 +08:00
e2eContainer := opts.FrontendContainer.
WithWorkdir("/src").
WithDirectory("/src", opts.HostSrc).
WithMountedCache(".nx", d.CacheVolume("nx-cache")).
WithEnvVariable("CI", "true").
WithEnvVariable("GRAFANA_URL", fmt.Sprintf("http://%s:%d", grafanaHost, grafanaPort)).
E2E: migrate to playwright (#107241) * separate playwright into its own folder * better separation * add login scenario, add tags * remove ui option * update CODEOWNERS * add a panels suite test * migrate queries test to playwright * rename + add dashlist test * add panelEdit_base * add geomap map controls test * add geomap-layer-types * add geomap-spatial-operations tests * add frontend-sandbox-panel tests * add smoke-tests-suite * add comment about adding datasource * add dashboard-browse-nested * add dashboard-browse * add dashboard-export-json * add dashboard-keybindings test * remove @wip tag * turn on screenshots and add comment for why this test fails * add dashboard-links-without-slug test * try adding permissions in the test as well * add dashboard-live-streaming * context in the test doesn't work - sad * create dashboard-public-templating * add dashboard-public-create and make live streaming more resilient * add share externally test * add dashboard-share-internally * add share-snapshot-create test * add dashboard-templating * add timepicker tests * add embedded-dashboard test * add general_dashboards test * add import-dashboard test * add load-options-from-url test * add new-constant-variable test * add custom-variable test * add new-datasource-variable test * add new-interval-variable test * add text-box-variable test * add new-query-variable test * add horizontal repeat test * add panel-vertical-repeat test * add empty-row-repeat test * add set-options-from-ui test * add snapshot-create test * add templating test * add textbox-variables test * add cloud-plugins-suite * add storybook verification tests * add playwright storybook verification workflow * add playwright browsers * update CODEOWNERS * test change to trigger storybook verification workflows * try container instead * get the version right... * go back to installing - less chance of forgetting to update * Basic Github Actions Squashed commit of the following: commit f84c650a71bba339da7f96303292717569d4caa6 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 13:23:46 2025 +0100 add arg for sharding, but not using it yet commit 7bcf0512c6652e079757e7f892d1802eb255a6a7 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:30:30 2025 +0100 less newline commit b6439118829a09a4c8c49d7692d38df0dff7e7ae Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:24:31 2025 +0100 less logs commit 38f871e9c235344f12ef31121fdfced549c5323c Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 10:00:26 2025 +0100 fix yaml commit db9a773136eff40a82beb3b2f55f73ab0d07a717 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:57:47 2025 +0100 clean up files commit c0525f41fa8b498f2fdf71b5047301b0e2945498 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:44:56 2025 +0100 gha workflow commit 895bea7c527bfbb148a421b9283fa0805f6ab11d Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 19:33:08 2025 +0100 working dagger commit cea1f8443738e9e4b375c065dd4a1e466036bb74 Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 16:17:46 2025 +0100 wip * shard gha * some tidy up * add flags for exporting results, and a gha step to merge runs * fix shard gha * add dashboard-duplicate-panel test * add dashboard-outline test * add dashboards-add-panel * remove some commented out code * add dashboards-title-description test * add dashboards-remove-panel * don't install cypress * gha: check playwright results * add dashboards-edit-adhoc-variables test * fix check-jobs * add dagger cloud token * add dagger cloud token * add edit-datasource-variable test * update CODEOWNERS * add dashboards-edit-group-by-variables (skipped for now) * add dashboards-edit-panel-title-description test * add dashboards-edit-transparent-bg test * add dashboards-edit-query-variables test * run with 8 shards * add dashboards-edit-variables * tidy up gha * add dashboard-group-panels * fix action * try to cache the grafana build * fix missing action becuase no checkout, use builtin continue-on-error instead * fix missing id * cat out.txt * debug build cache * fix debug build cache * add dashboards-panel-layouts test * tidy up * no more debug * fix grafana dir * add dashboards-move-panel test * skip some failing tests * mark up plugins tests with @plugins tag, only run @plugins tests in drone * Hackathon/Playwright Conversion - Various Suite (#107516) * Playwright Migration: Various Suite tests * skipping bad tests * fix some tests that can fail * fix uid * separate user for the verify-i18n test * build test plugins for grafana server * properly blur input fields * login manually * get dashboardPage from goto * ignore a couple of type assertions * remove a couple of timeouts * remove timeouts on dashboard-share-internally * use toBeHidden * make dashboard-share-internally more stable * remove TEMP_DAGGER_TOKEN * clean up visaulization-suggestions * unskip gauge test * unskip trace-view-scrolling * attempt to make create variable utils stable * unskip loki tests * make go linter happy * unskip edit-group-by-variables test * unskip move panel tests * isolate dashboard-timepicker tests with separate user * create data source as part of smoke test * make sure we're awaiting in dashboard-edit-adhoc-variables * make dashboards-edit-variables test more robust * Hackathon Playwright: Dashboards Search (#107580) * Hackathon Playwright: Dashboards Search * Feedback changes * make trace-view-scrolling more stable * add json report and bench step * fix bench version * move fail step to after the playwright report so we can report test failures * fix output file name * fix typo * try wrap in expect.poll * stability * bit more tidy up * fix dashboard-new-layouts tests * move test-plugins to e2e-playwright * fix go code for drone e2e run * move loki plugin-e2e test * make v2 dashboards work again --------- Co-authored-by: joshhunt <josh.hunt@grafana.com> Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com> Co-authored-by: Collin Fingar <collin.fingar@grafana.com> Co-authored-by: Jeff Levin <jeff@levinology.com>
2025-07-11 17:31:33 +08:00
WithServiceBinding(grafanaHost, opts.GrafanaService).
WithEnvVariable("bustcache", "1").
WithEnvVariable("PLAYWRIGHT_HTML_OPEN", "never").
WithEnvVariable("PLAYWRIGHT_HTML_OUTPUT_DIR", htmlResultsDir).
WithEnvVariable("PLAYWRIGHT_BLOB_OUTPUT_DIR", blobResultsDir)
if opts.CloudPluginCreds != nil {
fmt.Println("DEBUG: CloudPluginCreds file is provided, mounting to /tmp/outputs.json")
e2eContainer = e2eContainer.WithMountedFile("/tmp/outputs.json", opts.CloudPluginCreds)
}
e2eContainer = e2eContainer.WithExec(playwrightCommand, dagger.ContainerWithExecOpts{
Expect: dagger.ReturnTypeAny,
})
E2E: migrate to playwright (#107241) * separate playwright into its own folder * better separation * add login scenario, add tags * remove ui option * update CODEOWNERS * add a panels suite test * migrate queries test to playwright * rename + add dashlist test * add panelEdit_base * add geomap map controls test * add geomap-layer-types * add geomap-spatial-operations tests * add frontend-sandbox-panel tests * add smoke-tests-suite * add comment about adding datasource * add dashboard-browse-nested * add dashboard-browse * add dashboard-export-json * add dashboard-keybindings test * remove @wip tag * turn on screenshots and add comment for why this test fails * add dashboard-links-without-slug test * try adding permissions in the test as well * add dashboard-live-streaming * context in the test doesn't work - sad * create dashboard-public-templating * add dashboard-public-create and make live streaming more resilient * add share externally test * add dashboard-share-internally * add share-snapshot-create test * add dashboard-templating * add timepicker tests * add embedded-dashboard test * add general_dashboards test * add import-dashboard test * add load-options-from-url test * add new-constant-variable test * add custom-variable test * add new-datasource-variable test * add new-interval-variable test * add text-box-variable test * add new-query-variable test * add horizontal repeat test * add panel-vertical-repeat test * add empty-row-repeat test * add set-options-from-ui test * add snapshot-create test * add templating test * add textbox-variables test * add cloud-plugins-suite * add storybook verification tests * add playwright storybook verification workflow * add playwright browsers * update CODEOWNERS * test change to trigger storybook verification workflows * try container instead * get the version right... * go back to installing - less chance of forgetting to update * Basic Github Actions Squashed commit of the following: commit f84c650a71bba339da7f96303292717569d4caa6 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 13:23:46 2025 +0100 add arg for sharding, but not using it yet commit 7bcf0512c6652e079757e7f892d1802eb255a6a7 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:30:30 2025 +0100 less newline commit b6439118829a09a4c8c49d7692d38df0dff7e7ae Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:24:31 2025 +0100 less logs commit 38f871e9c235344f12ef31121fdfced549c5323c Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 10:00:26 2025 +0100 fix yaml commit db9a773136eff40a82beb3b2f55f73ab0d07a717 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:57:47 2025 +0100 clean up files commit c0525f41fa8b498f2fdf71b5047301b0e2945498 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:44:56 2025 +0100 gha workflow commit 895bea7c527bfbb148a421b9283fa0805f6ab11d Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 19:33:08 2025 +0100 working dagger commit cea1f8443738e9e4b375c065dd4a1e466036bb74 Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 16:17:46 2025 +0100 wip * shard gha * some tidy up * add flags for exporting results, and a gha step to merge runs * fix shard gha * add dashboard-duplicate-panel test * add dashboard-outline test * add dashboards-add-panel * remove some commented out code * add dashboards-title-description test * add dashboards-remove-panel * don't install cypress * gha: check playwright results * add dashboards-edit-adhoc-variables test * fix check-jobs * add dagger cloud token * add dagger cloud token * add edit-datasource-variable test * update CODEOWNERS * add dashboards-edit-group-by-variables (skipped for now) * add dashboards-edit-panel-title-description test * add dashboards-edit-transparent-bg test * add dashboards-edit-query-variables test * run with 8 shards * add dashboards-edit-variables * tidy up gha * add dashboard-group-panels * fix action * try to cache the grafana build * fix missing action becuase no checkout, use builtin continue-on-error instead * fix missing id * cat out.txt * debug build cache * fix debug build cache * add dashboards-panel-layouts test * tidy up * no more debug * fix grafana dir * add dashboards-move-panel test * skip some failing tests * mark up plugins tests with @plugins tag, only run @plugins tests in drone * Hackathon/Playwright Conversion - Various Suite (#107516) * Playwright Migration: Various Suite tests * skipping bad tests * fix some tests that can fail * fix uid * separate user for the verify-i18n test * build test plugins for grafana server * properly blur input fields * login manually * get dashboardPage from goto * ignore a couple of type assertions * remove a couple of timeouts * remove timeouts on dashboard-share-internally * use toBeHidden * make dashboard-share-internally more stable * remove TEMP_DAGGER_TOKEN * clean up visaulization-suggestions * unskip gauge test * unskip trace-view-scrolling * attempt to make create variable utils stable * unskip loki tests * make go linter happy * unskip edit-group-by-variables test * unskip move panel tests * isolate dashboard-timepicker tests with separate user * create data source as part of smoke test * make sure we're awaiting in dashboard-edit-adhoc-variables * make dashboards-edit-variables test more robust * Hackathon Playwright: Dashboards Search (#107580) * Hackathon Playwright: Dashboards Search * Feedback changes * make trace-view-scrolling more stable * add json report and bench step * fix bench version * move fail step to after the playwright report so we can report test failures * fix output file name * fix typo * try wrap in expect.poll * stability * bit more tidy up * fix dashboard-new-layouts tests * move test-plugins to e2e-playwright * fix go code for drone e2e run * move loki plugin-e2e test * make v2 dashboards work again --------- Co-authored-by: joshhunt <josh.hunt@grafana.com> Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com> Co-authored-by: Collin Fingar <collin.fingar@grafana.com> Co-authored-by: Jeff Levin <jeff@levinology.com>
2025-07-11 17:31:33 +08:00
if opts.TestResultsExportDir != "" {
_, err := e2eContainer.Directory(testResultsDir).Export(ctx, opts.TestResultsExportDir)
if err != nil {
return nil, err
}
}
if opts.HTMLReportExportDir != "" {
_, err := e2eContainer.Directory(htmlResultsDir).Export(ctx, opts.HTMLReportExportDir)
if err != nil {
return nil, err
}
}
if opts.BlobReportExportDir != "" {
_, err := e2eContainer.Directory(blobResultsDir).Export(ctx, opts.BlobReportExportDir)
if err != nil {
return nil, err
}
}
return e2eContainer, nil
}
func buildPlaywrightCommand(opts RunTestOpts) []string {
playwrightReporters := []string{
"dot", // minimal output in shards
}
if opts.HTMLReportExportDir != "" {
playwrightReporters = append(playwrightReporters, "html")
}
if opts.BlobReportExportDir != "" {
playwrightReporters = append(playwrightReporters, "blob")
}
playwrightExec := strings.Split(opts.PlaywrightCommand, " ")
playwrightCommand := append(playwrightExec,
E2E: migrate to playwright (#107241) * separate playwright into its own folder * better separation * add login scenario, add tags * remove ui option * update CODEOWNERS * add a panels suite test * migrate queries test to playwright * rename + add dashlist test * add panelEdit_base * add geomap map controls test * add geomap-layer-types * add geomap-spatial-operations tests * add frontend-sandbox-panel tests * add smoke-tests-suite * add comment about adding datasource * add dashboard-browse-nested * add dashboard-browse * add dashboard-export-json * add dashboard-keybindings test * remove @wip tag * turn on screenshots and add comment for why this test fails * add dashboard-links-without-slug test * try adding permissions in the test as well * add dashboard-live-streaming * context in the test doesn't work - sad * create dashboard-public-templating * add dashboard-public-create and make live streaming more resilient * add share externally test * add dashboard-share-internally * add share-snapshot-create test * add dashboard-templating * add timepicker tests * add embedded-dashboard test * add general_dashboards test * add import-dashboard test * add load-options-from-url test * add new-constant-variable test * add custom-variable test * add new-datasource-variable test * add new-interval-variable test * add text-box-variable test * add new-query-variable test * add horizontal repeat test * add panel-vertical-repeat test * add empty-row-repeat test * add set-options-from-ui test * add snapshot-create test * add templating test * add textbox-variables test * add cloud-plugins-suite * add storybook verification tests * add playwright storybook verification workflow * add playwright browsers * update CODEOWNERS * test change to trigger storybook verification workflows * try container instead * get the version right... * go back to installing - less chance of forgetting to update * Basic Github Actions Squashed commit of the following: commit f84c650a71bba339da7f96303292717569d4caa6 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 13:23:46 2025 +0100 add arg for sharding, but not using it yet commit 7bcf0512c6652e079757e7f892d1802eb255a6a7 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:30:30 2025 +0100 less newline commit b6439118829a09a4c8c49d7692d38df0dff7e7ae Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:24:31 2025 +0100 less logs commit 38f871e9c235344f12ef31121fdfced549c5323c Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 10:00:26 2025 +0100 fix yaml commit db9a773136eff40a82beb3b2f55f73ab0d07a717 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:57:47 2025 +0100 clean up files commit c0525f41fa8b498f2fdf71b5047301b0e2945498 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:44:56 2025 +0100 gha workflow commit 895bea7c527bfbb148a421b9283fa0805f6ab11d Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 19:33:08 2025 +0100 working dagger commit cea1f8443738e9e4b375c065dd4a1e466036bb74 Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 16:17:46 2025 +0100 wip * shard gha * some tidy up * add flags for exporting results, and a gha step to merge runs * fix shard gha * add dashboard-duplicate-panel test * add dashboard-outline test * add dashboards-add-panel * remove some commented out code * add dashboards-title-description test * add dashboards-remove-panel * don't install cypress * gha: check playwright results * add dashboards-edit-adhoc-variables test * fix check-jobs * add dagger cloud token * add dagger cloud token * add edit-datasource-variable test * update CODEOWNERS * add dashboards-edit-group-by-variables (skipped for now) * add dashboards-edit-panel-title-description test * add dashboards-edit-transparent-bg test * add dashboards-edit-query-variables test * run with 8 shards * add dashboards-edit-variables * tidy up gha * add dashboard-group-panels * fix action * try to cache the grafana build * fix missing action becuase no checkout, use builtin continue-on-error instead * fix missing id * cat out.txt * debug build cache * fix debug build cache * add dashboards-panel-layouts test * tidy up * no more debug * fix grafana dir * add dashboards-move-panel test * skip some failing tests * mark up plugins tests with @plugins tag, only run @plugins tests in drone * Hackathon/Playwright Conversion - Various Suite (#107516) * Playwright Migration: Various Suite tests * skipping bad tests * fix some tests that can fail * fix uid * separate user for the verify-i18n test * build test plugins for grafana server * properly blur input fields * login manually * get dashboardPage from goto * ignore a couple of type assertions * remove a couple of timeouts * remove timeouts on dashboard-share-internally * use toBeHidden * make dashboard-share-internally more stable * remove TEMP_DAGGER_TOKEN * clean up visaulization-suggestions * unskip gauge test * unskip trace-view-scrolling * attempt to make create variable utils stable * unskip loki tests * make go linter happy * unskip edit-group-by-variables test * unskip move panel tests * isolate dashboard-timepicker tests with separate user * create data source as part of smoke test * make sure we're awaiting in dashboard-edit-adhoc-variables * make dashboards-edit-variables test more robust * Hackathon Playwright: Dashboards Search (#107580) * Hackathon Playwright: Dashboards Search * Feedback changes * make trace-view-scrolling more stable * add json report and bench step * fix bench version * move fail step to after the playwright report so we can report test failures * fix output file name * fix typo * try wrap in expect.poll * stability * bit more tidy up * fix dashboard-new-layouts tests * move test-plugins to e2e-playwright * fix go code for drone e2e run * move loki plugin-e2e test * make v2 dashboards work again --------- Co-authored-by: joshhunt <josh.hunt@grafana.com> Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com> Co-authored-by: Collin Fingar <collin.fingar@grafana.com> Co-authored-by: Jeff Levin <jeff@levinology.com>
2025-07-11 17:31:33 +08:00
"--reporter",
strings.Join(playwrightReporters, ","),
"--output",
testResultsDir,
)
E2E: migrate to playwright (#107241) * separate playwright into its own folder * better separation * add login scenario, add tags * remove ui option * update CODEOWNERS * add a panels suite test * migrate queries test to playwright * rename + add dashlist test * add panelEdit_base * add geomap map controls test * add geomap-layer-types * add geomap-spatial-operations tests * add frontend-sandbox-panel tests * add smoke-tests-suite * add comment about adding datasource * add dashboard-browse-nested * add dashboard-browse * add dashboard-export-json * add dashboard-keybindings test * remove @wip tag * turn on screenshots and add comment for why this test fails * add dashboard-links-without-slug test * try adding permissions in the test as well * add dashboard-live-streaming * context in the test doesn't work - sad * create dashboard-public-templating * add dashboard-public-create and make live streaming more resilient * add share externally test * add dashboard-share-internally * add share-snapshot-create test * add dashboard-templating * add timepicker tests * add embedded-dashboard test * add general_dashboards test * add import-dashboard test * add load-options-from-url test * add new-constant-variable test * add custom-variable test * add new-datasource-variable test * add new-interval-variable test * add text-box-variable test * add new-query-variable test * add horizontal repeat test * add panel-vertical-repeat test * add empty-row-repeat test * add set-options-from-ui test * add snapshot-create test * add templating test * add textbox-variables test * add cloud-plugins-suite * add storybook verification tests * add playwright storybook verification workflow * add playwright browsers * update CODEOWNERS * test change to trigger storybook verification workflows * try container instead * get the version right... * go back to installing - less chance of forgetting to update * Basic Github Actions Squashed commit of the following: commit f84c650a71bba339da7f96303292717569d4caa6 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 13:23:46 2025 +0100 add arg for sharding, but not using it yet commit 7bcf0512c6652e079757e7f892d1802eb255a6a7 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:30:30 2025 +0100 less newline commit b6439118829a09a4c8c49d7692d38df0dff7e7ae Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 12:24:31 2025 +0100 less logs commit 38f871e9c235344f12ef31121fdfced549c5323c Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 10:00:26 2025 +0100 fix yaml commit db9a773136eff40a82beb3b2f55f73ab0d07a717 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:57:47 2025 +0100 clean up files commit c0525f41fa8b498f2fdf71b5047301b0e2945498 Author: joshhunt <josh.hunt@grafana.com> Date: Tue Jul 1 09:44:56 2025 +0100 gha workflow commit 895bea7c527bfbb148a421b9283fa0805f6ab11d Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 19:33:08 2025 +0100 working dagger commit cea1f8443738e9e4b375c065dd4a1e466036bb74 Author: joshhunt <josh.hunt@grafana.com> Date: Mon Jun 30 16:17:46 2025 +0100 wip * shard gha * some tidy up * add flags for exporting results, and a gha step to merge runs * fix shard gha * add dashboard-duplicate-panel test * add dashboard-outline test * add dashboards-add-panel * remove some commented out code * add dashboards-title-description test * add dashboards-remove-panel * don't install cypress * gha: check playwright results * add dashboards-edit-adhoc-variables test * fix check-jobs * add dagger cloud token * add dagger cloud token * add edit-datasource-variable test * update CODEOWNERS * add dashboards-edit-group-by-variables (skipped for now) * add dashboards-edit-panel-title-description test * add dashboards-edit-transparent-bg test * add dashboards-edit-query-variables test * run with 8 shards * add dashboards-edit-variables * tidy up gha * add dashboard-group-panels * fix action * try to cache the grafana build * fix missing action becuase no checkout, use builtin continue-on-error instead * fix missing id * cat out.txt * debug build cache * fix debug build cache * add dashboards-panel-layouts test * tidy up * no more debug * fix grafana dir * add dashboards-move-panel test * skip some failing tests * mark up plugins tests with @plugins tag, only run @plugins tests in drone * Hackathon/Playwright Conversion - Various Suite (#107516) * Playwright Migration: Various Suite tests * skipping bad tests * fix some tests that can fail * fix uid * separate user for the verify-i18n test * build test plugins for grafana server * properly blur input fields * login manually * get dashboardPage from goto * ignore a couple of type assertions * remove a couple of timeouts * remove timeouts on dashboard-share-internally * use toBeHidden * make dashboard-share-internally more stable * remove TEMP_DAGGER_TOKEN * clean up visaulization-suggestions * unskip gauge test * unskip trace-view-scrolling * attempt to make create variable utils stable * unskip loki tests * make go linter happy * unskip edit-group-by-variables test * unskip move panel tests * isolate dashboard-timepicker tests with separate user * create data source as part of smoke test * make sure we're awaiting in dashboard-edit-adhoc-variables * make dashboards-edit-variables test more robust * Hackathon Playwright: Dashboards Search (#107580) * Hackathon Playwright: Dashboards Search * Feedback changes * make trace-view-scrolling more stable * add json report and bench step * fix bench version * move fail step to after the playwright report so we can report test failures * fix output file name * fix typo * try wrap in expect.poll * stability * bit more tidy up * fix dashboard-new-layouts tests * move test-plugins to e2e-playwright * fix go code for drone e2e run * move loki plugin-e2e test * make v2 dashboards work again --------- Co-authored-by: joshhunt <josh.hunt@grafana.com> Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com> Co-authored-by: Collin Fingar <collin.fingar@grafana.com> Co-authored-by: Jeff Levin <jeff@levinology.com>
2025-07-11 17:31:33 +08:00
if opts.Shard != "" {
playwrightCommand = append(playwrightCommand, "--shard", opts.Shard)
}
return playwrightCommand
}