feat: Allow running Playwright e2e tests using Vite dev server (no-changelog) #47478
ci-pull-requests.yml
on: pull_request
Install & Build
1m 30s
Matrix: E2E Tests / Multi-Main: Isolated / test
Matrix: E2E Tests / Multi-Main: UI / test
Matrix: Unit tests / unit-test-frontend
Unit tests
/
Backend Unit Tests
5m 3s
Unit tests
/
Backend Integration Tests
5m 21s
Unit tests
/
Nodes Unit Tests
42s
Bundle Size Check
Required Checks
4s
Annotations
19 errors, 12 warnings, and 9 notices
|
E2E Tests / Multi-Main: UI / Test (Shard 2/8)
Process completed with exit code 1.
|
|
[multi-main:ui] › tests/ui/18-user-service.spec.ts:4:6 › User API Service › should create a user with default values:
packages/testing/playwright/services/api-helper.ts#L253
3) [multi-main:ui] › tests/ui/18-user-service.spec.ts:4:6 › User API Service › should create a user with default values
TestError: {"code":401,"message":"Wrong username or password. Do you have caps lock on?","stacktrace":"ResponseError: Wrong username or password. Do you have caps lock on?\n at AuthController.login (/usr/local/lib/node_modules/n8n/src/controllers/auth.controller.ts:121:9)\n at processTicksAndRejections (node:internal/process/task_queues:105:5)\n at handler (/usr/local/lib/node_modules/n8n/src/controller.registry.ts:95:12)\n at /usr/local/lib/node_modules/n8n/src/response-helper.ts:161:17"}
at services/api-helper.ts:253
251 | if (!response.ok()) {
252 | const errorText = await response.text();
> 253 | throw new TestError(errorText);
| ^
254 | }
255 |
256 | let responseData: unknown;
at ApiHelpers.loginAndSetCookies (/home/runner/_work/n8n/n8n/packages/testing/playwright/services/api-helper.ts:253:10)
at ApiHelpers.signin (/home/runner/_work/n8n/n8n/packages/testing/playwright/services/api-helper.ts:145:10)
at ApiHelpers.setupFromTags (/home/runner/_work/n8n/n8n/packages/testing/playwright/services/api-helper.ts:86:11)
at Object.api (/home/runner/_work/n8n/n8n/packages/testing/playwright/fixtures/base.ts:233:3)
|
|
[multi-main:ui] › tests/ui/18-user-management.spec.ts:130:6 › User Management › shouldn't allow users to set invalid email:
packages/testing/playwright/services/api-helper.ts#L253
2) [multi-main:ui] › tests/ui/18-user-management.spec.ts:130:6 › User Management › shouldn't allow users to set invalid email
TestError: {"code":401,"message":"Wrong username or password. Do you have caps lock on?","stacktrace":"ResponseError: Wrong username or password. Do you have caps lock on?\n at AuthController.login (/usr/local/lib/node_modules/n8n/src/controllers/auth.controller.ts:121:9)\n at handler (/usr/local/lib/node_modules/n8n/src/controller.registry.ts:95:12)\n at /usr/local/lib/node_modules/n8n/src/response-helper.ts:161:17"}
at services/api-helper.ts:253
251 | if (!response.ok()) {
252 | const errorText = await response.text();
> 253 | throw new TestError(errorText);
| ^
254 | }
255 |
256 | let responseData: unknown;
at ApiHelpers.loginAndSetCookies (/home/runner/_work/n8n/n8n/packages/testing/playwright/services/api-helper.ts:253:10)
at ApiHelpers.signin (/home/runner/_work/n8n/n8n/packages/testing/playwright/services/api-helper.ts:145:10)
at ApiHelpers.setupFromTags (/home/runner/_work/n8n/n8n/packages/testing/playwright/services/api-helper.ts:86:11)
at Object.n8n (/home/runner/_work/n8n/n8n/packages/testing/playwright/fixtures/base.ts:202:3)
|
|
[multi-main:ui] › tests/ui/18-user-management.spec.ts:17:6 › User Management › should prevent non-owners to access UM settings:
packages/testing/playwright/tests/ui/18-user-management.spec.ts#L21
1) [multi-main:ui] › tests/ui/18-user-management.spec.ts:17:6 › User Management › should prevent non-owners to access UM settings
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByTestId('new-workflow-card')
Expected: visible
Timeout: 10000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 10000ms
- waiting for getByTestId('new-workflow-card')
19 | await n8n.api.users.create();
20 | await n8n.navigate.toUsers();
> 21 | await expect(n8n.workflows.getNewWorkflowCard()).toBeVisible();
| ^
22 | });
23 |
24 | test('should allow instance owner to access UM settings', async ({ n8n }) => {
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/18-user-management.spec.ts:21:52
|
|
[multi-main:ui] › tests/ui/18-user-management.spec.ts:17:6 › User Management › should prevent non-owners to access UM settings:
packages/testing/playwright/tests/ui/18-user-management.spec.ts#L21
1) [multi-main:ui] › tests/ui/18-user-management.spec.ts:17:6 › User Management › should prevent non-owners to access UM settings
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByTestId('new-workflow-card')
Expected: visible
Timeout: 10000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 10000ms
- waiting for getByTestId('new-workflow-card')
19 | await n8n.api.users.create();
20 | await n8n.navigate.toUsers();
> 21 | await expect(n8n.workflows.getNewWorkflowCard()).toBeVisible();
| ^
22 | });
23 |
24 | test('should allow instance owner to access UM settings', async ({ n8n }) => {
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/18-user-management.spec.ts:21:52
|
|
[multi-main:ui] › tests/ui/18-user-management.spec.ts:17:6 › User Management › should prevent non-owners to access UM settings:
packages/testing/playwright/tests/ui/18-user-management.spec.ts#L21
1) [multi-main:ui] › tests/ui/18-user-management.spec.ts:17:6 › User Management › should prevent non-owners to access UM settings
Error: expect(locator).toBeVisible() failed
Locator: getByTestId('new-workflow-card')
Expected: visible
Timeout: 10000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 10000ms
- waiting for getByTestId('new-workflow-card')
19 | await n8n.api.users.create();
20 | await n8n.navigate.toUsers();
> 21 | await expect(n8n.workflows.getNewWorkflowCard()).toBeVisible();
| ^
22 | });
23 |
24 | test('should allow instance owner to access UM settings', async ({ n8n }) => {
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/18-user-management.spec.ts:21:52
|
|
E2E Tests / Multi-Main: Isolated / Test (Shard 1/1)
Process completed with exit code 1.
|
|
[multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:66:6 › Workflow Viewer Permissions @isolated › user without workflow:update scope cannot drag nodes @auth:owner:
packages/testing/playwright/tests/ui/56-workflow-viewer-permissions.spec.ts#L82
3) [multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:66:6 › Workflow Viewer Permissions @isolated › user without workflow:update scope cannot drag nodes @auth:owner
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(received).toBe(expected) // Object.is equality
Expected: 924
Received: 1020
80 |
81 | // Position should remain unchanged
> 82 | expect(finalPosition?.x).toBe(initialPosition?.x);
| ^
83 | expect(finalPosition?.y).toBe(initialPosition?.y);
84 | });
85 |
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/56-workflow-viewer-permissions.spec.ts:82:28
|
|
[multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:66:6 › Workflow Viewer Permissions @isolated › user without workflow:update scope cannot drag nodes @auth:owner:
packages/testing/playwright/tests/ui/56-workflow-viewer-permissions.spec.ts#L82
3) [multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:66:6 › Workflow Viewer Permissions @isolated › user without workflow:update scope cannot drag nodes @auth:owner
Error: expect(received).toBe(expected) // Object.is equality
Expected: 924
Received: 1020
80 |
81 | // Position should remain unchanged
> 82 | expect(finalPosition?.x).toBe(initialPosition?.x);
| ^
83 | expect(finalPosition?.y).toBe(initialPosition?.y);
84 | });
85 |
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/56-workflow-viewer-permissions.spec.ts:82:28
|
|
[multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:86:6 › Workflow Viewer Permissions @isolated › user without workflow:update can copy but cannot paste @auth:owner:
packages/testing/playwright/tests/ui/56-workflow-viewer-permissions.spec.ts#L99
2) [multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:86:6 › Workflow Viewer Permissions @isolated › user without workflow:update can copy but cannot paste @auth:owner
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toHaveCount(expected) failed
Locator: getByTestId('canvas-node')
Expected: 2
Received: 3
Timeout: 10000ms
Call log:
- Expect "toHaveCount" with timeout 10000ms
- waiting for getByTestId('canvas-node')
14 × locator resolved to 3 elements
- unexpected value "3"
97 | const nodeCountBefore = await n8n.canvas.getCanvasNodes().count();
98 | await n8n.page.keyboard.press('ControlOrMeta+V');
> 99 | await expect(n8n.canvas.getCanvasNodes()).toHaveCount(nodeCountBefore);
| ^
100 | });
101 |
102 | test('user with workflow:update scope can drag and paste @auth:owner', async ({ n8n }) => {
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/56-workflow-viewer-permissions.spec.ts:99:45
|
|
[multi-main:ui:isolated] › tests/ui/17-sharing.spec.ts:51:7 › @isolated › Sharing - Workflow and Credential Sharing (Sequential) › should create C2:
packages/testing/playwright/pages/components/CredentialModal.ts#L157
1) [multi-main:ui:isolated] › tests/ui/17-sharing.spec.ts:51:7 › @isolated › Sharing - Workflow and Credential Sharing (Sequential) › should create C2, share C2 with U1 and U2, as U3
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
TimeoutError: locator.click: Timeout 20000ms exceeded.
Call log:
- waiting for locator('.el-popper[aria-hidden="false"]').getByText('[email protected]')
at pages/components/CredentialModal.ts:157
155 | async addUserToSharing(email: string): Promise<void> {
156 | await this.getUsersSelect().click();
> 157 | await this.getVisibleDropdown().getByText(email.toLowerCase(), { exact: false }).click();
| ^
158 | }
159 |
160 | /**
at CredentialModal.addUserToSharing (/home/runner/_work/n8n/n8n/packages/testing/playwright/pages/components/CredentialModal.ts:157:84)
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/17-sharing.spec.ts:64:4
|
|
[multi-main:ui:isolated] › tests/ui/17-sharing.spec.ts:51:7 › @isolated › Sharing - Workflow and Credential Sharing (Sequential) › should create C2:
packages/testing/playwright/pages/components/CredentialModal.ts#L157
1) [multi-main:ui:isolated] › tests/ui/17-sharing.spec.ts:51:7 › @isolated › Sharing - Workflow and Credential Sharing (Sequential) › should create C2, share C2 with U1 and U2, as U3
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
TimeoutError: locator.click: Timeout 20000ms exceeded.
Call log:
- waiting for locator('.el-popper[aria-hidden="false"]').getByText('[email protected]')
at pages/components/CredentialModal.ts:157
155 | async addUserToSharing(email: string): Promise<void> {
156 | await this.getUsersSelect().click();
> 157 | await this.getVisibleDropdown().getByText(email.toLowerCase(), { exact: false }).click();
| ^
158 | }
159 |
160 | /**
at CredentialModal.addUserToSharing (/home/runner/_work/n8n/n8n/packages/testing/playwright/pages/components/CredentialModal.ts:157:84)
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/17-sharing.spec.ts:64:4
|
|
[multi-main:ui:isolated] › tests/ui/17-sharing.spec.ts:51:7 › @isolated › Sharing - Workflow and Credential Sharing (Sequential) › should create C2:
packages/testing/playwright/pages/components/CredentialModal.ts#L157
1) [multi-main:ui:isolated] › tests/ui/17-sharing.spec.ts:51:7 › @isolated › Sharing - Workflow and Credential Sharing (Sequential) › should create C2, share C2 with U1 and U2, as U3
TimeoutError: locator.click: Timeout 20000ms exceeded.
Call log:
- waiting for locator('.el-popper[aria-hidden="false"]').getByText('[email protected]')
at pages/components/CredentialModal.ts:157
155 | async addUserToSharing(email: string): Promise<void> {
156 | await this.getUsersSelect().click();
> 157 | await this.getVisibleDropdown().getByText(email.toLowerCase(), { exact: false }).click();
| ^
158 | }
159 |
160 | /**
at CredentialModal.addUserToSharing (/home/runner/_work/n8n/n8n/packages/testing/playwright/pages/components/CredentialModal.ts:157:84)
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/17-sharing.spec.ts:64:4
|
|
E2E Tests / Multi-Main: UI / Test (Shard 8/8)
Process completed with exit code 1.
|
|
[multi-main:ui] › tests/ui/pdf.spec.ts:4:6 › PDF Test › Can read and write PDF files and extract text:
packages/testing/playwright/tests/ui/pdf.spec.ts#L10
2) [multi-main:ui] › tests/ui/pdf.spec.ts:4:6 › PDF Test › Can read and write PDF files and extract text
Error: expect(locator).toBeVisible() failed
Locator: getByRole('alert').filter({ has: locator('.el-notification__title').filter({ hasText: 'Workflow executed successfully' }) })
Expected: visible
Timeout: 10000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 10000ms
- waiting for getByRole('alert').filter({ has: locator('.el-notification__title').filter({ hasText: 'Workflow executed successfully' }) })
8 | await expect(
9 | n8n.notifications.getNotificationByTitle('Workflow executed successfully'),
> 10 | ).toBeVisible();
| ^
11 | });
12 | });
13 |
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/pdf.spec.ts:10:5
|
|
[multi-main:ui] › tests/ui/task-runner-demo.spec.ts:28:6 › Task Runner Capability @capability:task-runner › should execute Python with task runner enabled:
packages/testing/playwright/pages/CanvasPage.ts#L146
1) [multi-main:ui] › tests/ui/task-runner-demo.spec.ts:28:6 › Task Runner Capability @capability:task-runner › should execute Python with task runner enabled
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
TimeoutError: locator.click: Timeout 20000ms exceeded.
Call log:
- waiting for getByTestId('node-creator-item-name').getByText('Code in Python (Native) (Beta)', { exact: true })
at pages/CanvasPage.ts:146
144 | await actionsCategory.click();
145 | }
> 146 | await this.nodeCreatorSubItem(options.action).click();
| ^
147 | } else if (options?.trigger) {
148 | // Check if Triggers category is collapsed and expand if needed
149 | const triggersCategory = this.page
at CanvasPage.addNode (/home/runner/_work/n8n/n8n/packages/testing/playwright/pages/CanvasPage.ts:146:50)
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/task-runner-demo.spec.ts:31:3
|
|
[multi-main:ui] › tests/ui/task-runner-demo.spec.ts:28:6 › Task Runner Capability @capability:task-runner › should execute Python with task runner enabled:
packages/testing/playwright/pages/CanvasPage.ts#L146
1) [multi-main:ui] › tests/ui/task-runner-demo.spec.ts:28:6 › Task Runner Capability @capability:task-runner › should execute Python with task runner enabled
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
TimeoutError: locator.click: Timeout 20000ms exceeded.
Call log:
- waiting for getByTestId('node-creator-item-name').getByText('Code in Python (Native) (Beta)', { exact: true })
at pages/CanvasPage.ts:146
144 | await actionsCategory.click();
145 | }
> 146 | await this.nodeCreatorSubItem(options.action).click();
| ^
147 | } else if (options?.trigger) {
148 | // Check if Triggers category is collapsed and expand if needed
149 | const triggersCategory = this.page
at CanvasPage.addNode (/home/runner/_work/n8n/n8n/packages/testing/playwright/pages/CanvasPage.ts:146:50)
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/task-runner-demo.spec.ts:31:3
|
|
[multi-main:ui] › tests/ui/task-runner-demo.spec.ts:28:6 › Task Runner Capability @capability:task-runner › should execute Python with task runner enabled:
packages/testing/playwright/pages/CanvasPage.ts#L146
1) [multi-main:ui] › tests/ui/task-runner-demo.spec.ts:28:6 › Task Runner Capability @capability:task-runner › should execute Python with task runner enabled
TimeoutError: locator.click: Timeout 20000ms exceeded.
Call log:
- waiting for getByTestId('node-creator-item-name').getByText('Code in Python (Native) (Beta)', { exact: true })
at pages/CanvasPage.ts:146
144 | await actionsCategory.click();
145 | }
> 146 | await this.nodeCreatorSubItem(options.action).click();
| ^
147 | } else if (options?.trigger) {
148 | // Check if Triggers category is collapsed and expand if needed
149 | const triggersCategory = this.page
at CanvasPage.addNode (/home/runner/_work/n8n/n8n/packages/testing/playwright/pages/CanvasPage.ts:146:50)
at /home/runner/_work/n8n/n8n/packages/testing/playwright/tests/ui/task-runner-demo.spec.ts:31:3
|
|
Required Checks
Process completed with exit code 1.
|
|
Unit tests / Backend Unit Tests
Unexpected 400 error: Unknown error
|
|
Unit tests / Backend Integration Tests
Unexpected 400 error: Unknown error
|
|
Unit tests / Frontend (1/2)
Unexpected 400 error: Unknown error
|
|
E2E Tests / Multi-Main: UI / Test (Shard 2/8)
Skipping sticky disk commit due to previous step failures
|
|
E2E Tests / Multi-Main: UI / Test (Shard 2/8)
- Step: unknown (failed)
|
|
E2E Tests / Multi-Main: UI / Test (Shard 2/8)
Found 2 failed/cancelled steps in previous workflow steps
|
|
E2E Tests / Multi-Main: Isolated / Test (Shard 1/1)
Skipping sticky disk commit due to previous step failures
|
|
E2E Tests / Multi-Main: Isolated / Test (Shard 1/1)
- Step: unknown (failed)
|
|
E2E Tests / Multi-Main: Isolated / Test (Shard 1/1)
Found 2 failed/cancelled steps in previous workflow steps
|
|
E2E Tests / Multi-Main: UI / Test (Shard 8/8)
Skipping sticky disk commit due to previous step failures
|
|
E2E Tests / Multi-Main: UI / Test (Shard 8/8)
- Step: unknown (failed)
|
|
E2E Tests / Multi-Main: UI / Test (Shard 8/8)
Found 2 failed/cancelled steps in previous workflow steps
|
|
🎭 Playwright Run Summary
2 skipped
71 passed (6.2m)
|
|
🎭 Playwright Run Summary
1 failed
[multi-main:ui] › tests/ui/18-user-management.spec.ts:17:6 › User Management › should prevent non-owners to access UM settings
2 flaky
[multi-main:ui] › tests/ui/18-user-management.spec.ts:130:6 › User Management › shouldn't allow users to set invalid email
[multi-main:ui] › tests/ui/18-user-service.spec.ts:4:6 › User API Service › should create a user with default values
21 skipped
49 passed (6.4m)
|
|
🎭 Playwright Run Summary
73 passed (6.7m)
|
|
🎭 Playwright Run Summary
8 skipped
65 passed (6.8m)
|
|
🎭 Playwright Run Summary
2 skipped
71 passed (7.3m)
|
|
🎭 Playwright Run Summary
2 skipped
71 passed (8.3m)
|
|
🎭 Playwright Run Summary
73 passed (7.9m)
|
|
🎭 Playwright Run Summary
2 failed
[multi-main:ui:isolated] › tests/ui/17-sharing.spec.ts:51:7 › @isolated › Sharing - Workflow and Credential Sharing (Sequential) › should create C2, share C2 with U1 and U2, as U3
[multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:86:6 › Workflow Viewer Permissions @isolated › user without workflow:update can copy but cannot paste @auth:owner
1 flaky
[multi-main:ui:isolated] › tests/ui/56-workflow-viewer-permissions.spec.ts:66:6 › Workflow Viewer Permissions @isolated › user without workflow:update scope cannot drag nodes @auth:owner
13 did not run
38 passed (9.7m)
|
|
🎭 Playwright Run Summary
1 failed
[multi-main:ui] › tests/ui/task-runner-demo.spec.ts:28:6 › Task Runner Capability @capability:task-runner › should execute Python with task runner enabled
1 flaky
[multi-main:ui] › tests/ui/pdf.spec.ts:4:6 › PDF Test › Can read and write PDF files and extract text
4 skipped
67 passed (10.0m)
|