46 Commits

Author	SHA1	Message	Date
Yury Semikhatsky	5f7db5bab4	fix: properly merge projects in defineConfig (#36898)	2025-08-01 14:13:28 -07:00
Jean-François Greffier	85a66912c6	feat(config): failOnFlakyTests option (#35109)	2025-03-11 10:06:20 -07:00
Simon Knott	aca3fb275b	fix(runner): validate `config.tsconfig` (#34849)	2025-02-20 09:11:50 +01:00
Adam Gastineau	aeed1f5909	fix(runner): display no projects error across all `test` modes (#34676)	2025-02-10 12:52:53 -08:00
Adam Gastineau	cb208836b5	fix: display error when project is not found (#34577)	2025-02-05 06:58:02 -08:00
Dmitry Gozman	b0fbe058ae	feat: TestProject.ignoreSnapshots (#30466)	2024-04-22 16:16:38 -07:00
Yury Semikhatsky	f3fde992eb	chore: remove --project-grep, add wildcard support (#29537) ... Reference #15128	2024-02-16 19:18:00 -08:00
Pavel Feldman	7aef5249d8	chore: pass the private data to the runner via config (#29467)	2024-02-13 09:34:03 -08:00
Yury Semikhatsky	7834beb932	chore: extract --project-grep option for matching with regex (#29445) ... Reference https://github.com/microsoft/playwright/issues/15128	2024-02-10 11:19:00 -08:00
Max Schmitt	a6e0af6767	feat: parse projects option as RegExp (#29327) ... Fixes https://github.com/microsoft/playwright/issues/15128 Relates https://github.com/microsoft/playwright/pull/17244 --------- Signed-off-by: Max Schmitt <max@schmitt.mx> Co-authored-by: Yury Semikhatsky <yurys@chromium.org>	2024-02-02 23:05:20 +01:00
Dmitry Gozman	d2dc8eb1e3	fix(components): make sure defineConfig(c1, c2) works (#28608)	2023-12-12 14:01:01 -08:00
Dmitry Gozman	62d4c3fe02	fix(defineConfig): do not add an empty project list to project-less configs (#28224) ... Otherwise, merging two configs without `projects` property will create a config with an empty project list, which is semantically different and always leads to "No tests found".	2023-11-17 13:36:50 -08:00
Pavel Feldman	08d6abab4a	chore: allow merging defineConfig (#26390)	2023-08-09 17:23:34 -07:00
Andrey Lushnikov	9af2b518f2	fix: exclude `tracesDir` option from `launchOptions` fixture (#24086) ... Fixes https://github.com/microsoft/playwright/issues/23645	2023-07-06 13:31:44 -07:00
Dmitry Gozman	91da67fab1	test: remove magic headers in ttest (#20867) ... Instead, explicitly import from '@playwright/test'.	2023-02-14 19:20:56 -08:00
chimurai	f72d400952	chore: fix typos (#20473)	2023-01-30 09:53:28 +01:00
Pavel Feldman	e08168e16e	chore: temporarily remove project and global setup, store (#20181)	2023-01-18 12:56:03 -08:00
Pavel Feldman	e065d608b6	chore: introduce defineConfig for easier JS typing (#20061) ... Fixes https://github.com/microsoft/playwright/issues/19694	2023-01-12 13:12:02 -08:00
Yury Semikhatsky	137070d889	Revert "chore: hide setup, store, TestProject.setupMatch, storageStat… (#19756) ... …eName (#19442)" This reverts commit 92dd734e04.	2022-12-28 15:39:31 -08:00
Yury Semikhatsky	92dd734e04	chore: hide setup, store, TestProject.setupMatch, storageStateName (#19442)	2022-12-13 22:48:38 -08:00
Yury Semikhatsky	beb15bb504	chore: rename setup => setupMatch (#19309)	2022-12-06 14:55:22 -08:00
Yury Semikhatsky	05fb3e6de8	Revert "chore: hide TestProject.setup and TestInfo.storage (#18800)" (#18836) ... Now that the patch has been cherry-picked to the release branch reverting it on main. This reverts commit 6deba5dc21.	2022-11-16 11:23:10 -08:00
Yury Semikhatsky	6deba5dc21	chore: hide TestProject.setup and TestInfo.storage (#18800)	2022-11-14 17:03:01 -08:00
Yury Semikhatsky	1d2fc1e963	feat(runner): project.setup (#18486)	2022-11-01 23:44:30 -07:00
Yury Semikhatsky	7337dd4e28	chore: revert project.{stage,run} (#18462)	2022-10-31 14:04:24 -07:00
Yury Semikhatsky	11eb719d13	feat(runner): project run: "always" (#18160) ... Projects marked with `run: 'always'` are non shard-able and run after failures.	2022-10-18 17:18:45 -07:00
Yury Semikhatsky	3b8f63d703	feat(runner): project.stopOnFailure (#18009)	2022-10-11 17:04:01 -07:00
Yury Semikhatsky	2d72d0ba03	chore: remove config.groups (#17974)	2022-10-10 16:42:48 -07:00
Yury Semikhatsky	9f17ee6871	feat: group filter (#17646)	2022-09-28 18:45:01 -07:00
Yury Semikhatsky	10d7c60abf	feat(runner): project execution schedule (#17548)	2022-09-23 20:01:27 -07:00
Jean-François Greffier	a15fe50e7b	feat(test runner): workers as percentage (#17400) ... Allows to set workers as a percentage of logical CPUs, for example "50%". Examples : ```bash npx playwright test --workers 3 npx playwright test --workers 50% ``` ```js const config: PlaywrightTestConfig = { // ... workers: '33%', } ```	2022-09-21 11:17:36 -07:00
Andrey Lushnikov	fafd9837ba	feat: introduce the `--ignore-snapshots` option (#17004) ... This patch introduces `--ignore-snapshots` Playwright Test CLI option, and `ignoreSnapshots` configuration argument.	2022-09-01 05:34:36 -07:00
Dmitry Gozman	7a86e140f5	chore: remove TestProject.projectSetup (#16321) ... There are better ideas to address this issue in more general manner.	2022-08-05 15:24:30 -07:00
Dmitry Gozman	3112edb4ca	feat(test runner): TestProject.projectSetup (#16063) ... `projectSetup` is a project-scoped alternative to `globalSetup`. It is only executed if at least one test from the project is scheduled to run.	2022-08-01 09:01:23 -07:00
Max Schmitt	004cd9273c	fix(test-runner): multiple webServer types (#15901)	2022-07-25 10:39:06 +02:00
Dmitry Gozman	b86926e691	fix(test runner): update timeout error messages (#15117) ... Some of the current timeout error messages are confusing, because they do not suggest that the issue is most likely a slow test. This PR updates timeout messages as follows: - Test timeout of 30000ms exceeded. - Test timeout of 30000ms exceeded while setting up "browser". - Test timeout of 30000ms exceeded while tearing down "context". - Test timeout of 30000ms exceeded while setting up "playwright configuration". - Test timeout of 30000ms exceeded while running "beforeEach" hook. - Test timeout of 30000ms exceeded while running "afterEach" hook. - "beforeAll" hook timeout of 30000ms exceeded. - "afterAll" hook timeout of 30000ms exceeded. - Worker teardown timeout of 30000ms exceeded. - "skip" modifier timeout of 30000ms exceeded. - Fixture "myCustomFixture" timeout of 5000ms exceeded.	2022-06-30 17:05:08 -07:00
Dmitry Gozman	91672595f2	fix(reporters): normalize usage of isTTY, env.CI and debug env variables (#12295) ... - `stdout.isTTY` controls whether list reporter updates lines or just adds them; - `env.CI` is used in a few places to affect the defaults: - whether to open interactive html; - default reporter dot/line; - default terminal reporter added to non-terminal reporters; - `env.PWTEST_SKIP_TEST_OUTPUT` is removed; - `env.PW_TEST_DEBUG_REPORTERS` is introduced specifically for tests.	2022-02-24 12:39:28 -08:00
Dmitry Gozman	d3c4323021	fix(test runner): improve error message for unexpected calls (#12240)	2022-02-18 18:25:18 -08:00
Dmitry Gozman	d9f849fb14	feat(test runner): replace declare/define with "options" (#10293) ... 1. Fixtures defined in test.extend() can now have `{ option: true }` configuration that makes them overridable in the config. Options support all other properties of fixtures - value/function, scope, auto. ``` const test = base.extend<MyOptions>({ foo: ['default', { option: true }], }); ``` 2. test.declare() and project.define are removed. 3. project.use applies overrides to default option values and nothing else. Any test.extend() and test.use() calls take priority over config options. Required user changes: if someone used to define fixture options with test.extend(), overriding them in config will stop working. The solution is to add `{ option: true }`. ``` // Old code export const test = base.extend<{ myOption: number, myFixture: number }>({ myOption: 123, myFixture: ({ myOption }, use) => use(2 * myOption), }); // New code export const test = base.extend<{ myOption: number, myFixture: number }>({ myOption: [123, { option: true }], myFixture: ({ myOption }, use) => use(2 * myOption), }); ```	2021-11-18 15:45:52 -08:00
Yury Semikhatsky	bafa426231	feat(runner): support multiple names in project filter (#8600)	2021-09-02 09:29:55 -07:00
Dmitry Gozman	8b2dd2e3d1	chore(test-runner): remove the notion of Spec (#7661) ... We now have Suites and Tests. When running multiple projects the whole suite is cloned for each project. Same happens for repeatEach. This simplifies the reporters API, but there is still room for improvement. JSON reporter continues to produce old json output.	2021-07-15 22:02:10 -07:00
Dmitry Gozman	6aefa02e91	feat(test runner): improve reporters api (#7370) ... - onEnd may return a Promise - onEnd now takes a result for the full run - onTimeout is replaced with onEnd(result)	2021-06-29 10:55:46 -07:00
Dmitry Gozman	4c6fa42810	fix(test runner): expose real stack traces and speed up locations (#7265) ... Stop wrapping/prepending error messages so that we do not loose the stack trace. For this, update a few manually thrown errors with better messages (usually including a file path). Speed up locations by doing manual `sourceMapSupport.wrapCallSite()` for a single call site. Performance gain in the runner process with 100 files x 100 tests each: - 25% on the fresh run without babel cache; - 80% on the cached run where babel is almost instant. Also some obvious cleanups around stack traces (removing unused code).	2021-06-23 10:30:54 -07:00
Pavel Feldman	184f2c2e93	feat(test-runner): allow specifying video size (#7158)	2021-06-16 07:51:54 -07:00
Dmitry Gozman	46a0213769	chore: remove internal uses of "folio" (#6931) ... Replaced by "pwt" or "playwright test".	2021-06-06 22:07:07 -07:00
Dmitry Gozman	f745bf1fbc	chore: bring in folio source (#6923) ... - Source now lives at `src/test`. - Former folio tests live at `tests/playwright-test`. - We use `src/test/internal.ts` that exposes base test without Playwright fixtures for most tests (to avoid modifications for now). - Test types live in `types/testFoo.d.ts`. - Stable test runner is installed to `tests/config/test-runner` during `npm install`. - All deps including test-only are now listed in `package.json`. Non-test deps must also be listed in `build_package.js` to get included.	2021-06-06 17:09:53 -07:00