root
/
playwright
mirror of https://github.com/microsoft/playwright.git
1
0
Fork 0
Code Issues Actions 1 Packages Projects Releases Wiki Activity
playwright/tests/playwright-test
History
Joe-Hendley 6ae5cd3824
feat: implement flag to fail flaky tests (#30618) 
Implements feature requested in
https://github.com/microsoft/playwright/issues/30457

The test runner treats flaky tests as failures when the flag is enabled,
but still reports flaky tests as flaky in the reporting interface. It
feels like something worth discussing as this behaviour makes sense to
me, but looked a bit odd to @BJSS-russell-pollock when I ran this past
him.

Closes #30457.
 		2024-05-15 09:10:10 -07:00
..
__screenshots__/playwright.artifacts.spec.ts feat: allow options in automatic screenshots (#19143) 2022-12-07 08:45:33 -08:00
assets chore: allow calling spawnSync on Node.js file inside test (#26429) 2023-08-11 18:37:14 +02:00
stable-test-runner chore: roll stable test runner to 1.44 beta (#30600) 2024-04-29 18:35:21 -07:00
access-data.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
babel.spec.ts chore: allow marking scripts as external for transform (#23449) 2023-06-01 20:28:49 -07:00
basic.spec.ts chore: rewrite worker main through TestInfoImpl._runAsStage (#29644) 2024-03-04 19:36:58 -08:00
clear-cache.spec.ts chore: fix typos (#30645) 2024-05-08 19:40:03 +01:00
command-line-filter.spec.ts chore: remove unnecessary args in ttest (#21012) 2023-02-18 13:08:17 -08:00
config.spec.ts feat: TestProject.ignoreSnapshots (#30466) 2024-04-22 16:16:38 -07:00
decorators.spec.ts chore: allow ts decorators (#22080) 2023-03-29 20:43:08 -07:00
deps.spec.ts chore(test runner): do not produce some of the fake skipped test results (#27730) 2023-10-20 17:01:46 -07:00
esm.spec.ts test(esm): fix import attribute tests (#30798) 2024-05-14 17:59:41 +01:00
exit-code.spec.ts feat: implement flag to fail flaky tests (#30618) 2024-05-15 09:10:10 -07:00
expect-configure.spec.ts chore: populate matcherResult in web assertions (#27133) 2023-09-16 14:24:10 -07:00
expect-poll.spec.ts chore: show last poller error on test timeout (#24292) 2023-07-18 17:03:26 -07:00
expect-soft.spec.ts docs: do not call custom expect message an "error message" (#29390) 2024-02-06 12:12:45 -08:00
expect-to-pass.spec.ts feat: Support intervals arg for toPass (#30037) 2024-03-29 10:17:29 -07:00
expect.spec.ts chore: fix typos (#30645) 2024-05-08 19:40:03 +01:00
find-related-tests.spec.ts chore: rename find-related-tests to find-related-test-files (#29483) 2024-02-13 15:40:49 -08:00
fixture-errors.spec.ts fix(test runner): update fixture teardown error message (#30109) 2024-03-25 17:04:03 -07:00
fixtures.spec.ts fix(test runner): make sure auto worker fixtures run before auto test fixtures (#27131) 2023-09-15 14:51:31 -07:00
gitignore.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
global-setup.spec.ts chore: get rid of ConfigInWorker, use FullConfig instead (#30517) 2024-04-24 14:14:05 -07:00
golden.spec.ts chore: roll stable test runner to 1.43.0-beta-1712217576000 (#30250) 2024-04-04 11:26:03 -07:00
hooks.spec.ts fix(reporters): properly determine flaky status for serial mode (#30529) 2024-04-25 13:39:14 -07:00
list-files.spec.ts chore: implement server-based list files (#29633) 2024-02-23 08:29:44 -08:00
list-mode.spec.ts Revert "fix: respect .only in --list mode" (#28770) 2023-12-22 11:03:30 -08:00
loader.spec.ts chore: unconflict ct test ports (#29332) 2024-02-05 16:47:39 -08:00
match-grep.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
max-failures.spec.ts chore(test runner): do not produce some of the fake skipped test results (#27730) 2023-10-20 17:01:46 -07:00
override-timeout.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
playwright-test-fixtures.ts feat: exposes tags in testInfo (#29794) 2024-03-06 08:33:06 -08:00
playwright.artifacts.spec.ts fix(last-run): remove globalOutputDir (#30571) 2024-04-29 09:02:54 -07:00
playwright.config.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
playwright.config.ts devops: custom blob report name for test runner tests (#29280) 2024-02-01 09:17:48 -08:00
playwright.connect.spec.ts Fix various typos (Fixes #27396) (#27391) 2023-10-04 19:56:42 -07:00
playwright.ct-build.spec.ts fix(ct): allow importing components from node_modules (#30493) 2024-04-24 12:32:28 -07:00
playwright.ct-react.spec.ts fix: pass key attribute from jsx to component test (#30426) 2024-04-26 09:49:44 -07:00
playwright.fetch.spec.ts test: unflake "should stop tracing on requestContext.dispose()" (#23489) 2023-06-02 16:37:50 -07:00
playwright.reuse.browser.spec.ts fix: encode launch options correctly when reusing browser (#24174) 2023-07-12 16:40:55 +02:00
playwright.reuse.spec.ts chore: do not close the reused context when video is on (#30807) 2024-05-15 09:05:06 -07:00
playwright.spec.ts feat: APIRequestContext dispose reason (#30765) 2024-05-13 18:51:30 -07:00
playwright.trace.spec.ts chore(test runner): move timeout handling to the top, stop inheriting runnable (#29857) 2024-03-08 15:19:36 -08:00
playwright.unhandled.spec.ts fix(test runner): properly handle uncaught errors in test.fail() (#27734) 2023-10-23 09:14:31 -07:00
repeat-each.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
reporter-attachment.spec.ts fix: quote trace file path when printing error message (#29052) 2024-01-18 14:23:12 -08:00
reporter-base.spec.ts test: relax "should print if globalTimeout is reached" expectation (#30558) 2024-04-25 18:40:58 -07:00
reporter-blob.spec.ts chore: fix typos (#30645) 2024-05-08 19:40:03 +01:00
reporter-dot.spec.ts chore: run more reporter tests through blob report, some fixes (#23765) 2023-06-16 21:30:55 -07:00
reporter-github.spec.ts chore: run more reporter tests through blob report, some fixes (#23765) 2023-06-16 21:30:55 -07:00
reporter-html.spec.ts fix: match client step and server action by explicit stepId (#30641) 2024-05-14 12:10:46 -07:00
reporter-json.spec.ts chore: add common env vars for junit and json reporters (#30611) 2024-05-01 10:16:49 -07:00
reporter-junit.spec.ts chore: add common env vars for junit and json reporters (#30611) 2024-05-01 10:16:49 -07:00
reporter-line.spec.ts chore: run more reporter tests through blob report, some fixes (#23765) 2023-06-16 21:30:55 -07:00
reporter-list.spec.ts chore: more network panel polish (#26780) 2023-08-29 22:20:28 -07:00
reporter-markdown.spec.ts fix(merge): normalize path separators when merging across platforms (#28227) 2023-11-27 12:43:56 -08:00
reporter-onend.spec.ts feat(onEnd): allow overriding the exit code (#27010) 2023-09-12 13:37:30 -07:00
reporter.spec.ts test: unflake "should work with custom reporter" (#30526) 2024-04-24 20:03:39 -07:00
resolver.spec.ts fix(tsconfig): when extending, retain pathsBasePath from the original config (#29822) 2024-03-05 16:34:39 -08:00
retry.spec.ts fix(reporters): properly determine flaky status for serial mode (#30529) 2024-04-25 13:39:14 -07:00
runner.spec.ts Revert "fix: shut down workers before reporter.onEnd (#30329)" (#30551) 2024-04-25 14:21:54 -07:00
shard.spec.ts Fix various typos (Fixes #27396) (#27391) 2023-10-04 19:56:42 -07:00
snapshot-path-template.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
stdio.spec.ts feat(test runner): do not mock tty in the worker process (#30107) 2024-03-25 15:31:58 -07:00
test-extend.spec.ts chore: composed->merge (#27555) 2023-10-11 13:56:27 -07:00
test-grep.spec.ts Fix various typos (Fixes #27396) (#27391) 2023-10-04 19:56:42 -07:00
test-ignore.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
test-info.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
test-modifiers.spec.ts fix(reporters): properly determine flaky status for serial mode (#30529) 2024-04-25 13:39:14 -07:00
test-output-dir.spec.ts chore: trim file names longer than 60 chars (was 100) (#29725) 2024-03-18 12:53:15 -07:00
test-parallel.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
test-serial.spec.ts fix(reporters): properly determine flaky status for serial mode (#30529) 2024-04-25 13:39:14 -07:00
test-step.spec.ts test: make expectations readable for more step reporting tests (#30468) 2024-04-22 15:30:51 -07:00
test-tag.spec.ts chore: fix typos (#30645) 2024-05-08 19:40:03 +01:00
test-use.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00
timeout.spec.ts chore(test runner): move timeout handling to the top, stop inheriting runnable (#29857) 2024-03-08 15:19:36 -08:00
to-have-screenshot.spec.ts chore: roll stable test runner to 1.43.0-beta-1712217576000 (#30250) 2024-04-04 11:26:03 -07:00
types-2.spec.ts feat(test runner): tags/annotations (#29248) 2024-02-07 16:31:25 -08:00
types.spec.ts fix(components): make sure defineConfig(c1, c2) works (#28608) 2023-12-12 14:01:01 -08:00
ui-mode-fixtures.ts feat(ui-mode): show native tags in test tree (#30092) 2024-03-26 01:06:22 +01:00
ui-mode-test-attachments.spec.ts fix(trace-viewer): encode attachment filenames as UTF-8 (#29993) 2024-03-18 21:26:45 +01:00
ui-mode-test-ct.spec.ts chore: automatically detect the dev server (#29176) 2024-01-25 11:46:47 -08:00
ui-mode-test-filters.spec.ts feat(ui-mode): show native tags in test tree (#30092) 2024-03-26 01:06:22 +01:00
ui-mode-test-output.spec.ts chore: implement faster ansi2html format for console (#26826) 2023-09-01 09:09:47 -07:00
ui-mode-test-progress.spec.ts fix(test runner): make TestTracing responsible for trace management (#29181) 2024-01-26 15:52:25 -08:00
ui-mode-test-run.spec.ts chore: fix total progress reporting w/ deps (#27716) 2023-10-19 20:07:47 -07:00
ui-mode-test-screencast.spec.ts chore: do not select after hooks automatically (#27805) 2023-10-25 17:05:06 -07:00
ui-mode-test-setup.spec.ts fix(ui): print config and global setup errors (#30531) 2024-04-24 18:54:48 -07:00
ui-mode-test-shortcut.spec.ts chore: address UI Mode keyboard shortcut feedback (#30088) 2024-03-25 19:48:20 +01:00
ui-mode-test-source.spec.ts chore: do not lose error name for js errors (#28177) 2023-11-15 18:27:32 -08:00
ui-mode-test-tree.spec.ts fix: rebuild project tree from scratch when listing tests (#30407) 2024-04-18 16:49:07 -07:00
ui-mode-test-update.spec.ts fix(ui-mode): do not loose run information after writing into testDir (#30312) 2024-04-10 19:01:51 +02:00
ui-mode-test-watch.spec.ts chore: prepare to reuse test server from ui mode (2) (#29966) 2024-03-18 09:59:02 -07:00
ui-mode-trace.spec.ts chore(test runner): move timeout handling to the top, stop inheriting runnable (#29857) 2024-03-08 15:19:36 -08:00
watch.spec.ts fix(test runner): avoid dependency tracking colliding between esm and cjs (#29994) 2024-03-18 17:17:58 -07:00
web-server.spec.ts fix(HEAD): revert GET->HEAD migration, net-effect was negative (#29738) 2024-02-29 09:02:05 -08:00
worker-index.spec.ts test: remove magic headers in ttest (#20867) 2023-02-14 19:20:56 -08:00