/writing-e2e-tests — How desktop E2E tests are written

An E2E test is worth its cost only when it fails for exactly one reason: the user-visible behavior it names is broken. It must not fail because the machine was slow, the theme changed, a sidebar moved, or a retry happened to pass.

Read /writing-tests first for the general rules (observable state, no mocks, migration ledgers). Read apps/desktop/e2e/README.md for the full fixture and project reference. This skill is the checklist that turns those into a test that stays green for the right reasons.

Choose the layer before writing anything

The behavior is…	Owning test
Pure computation (layout math, latch rules, range selection)	Pure unit test — extract the function first
Observable through editor state after a click, drag, or key	TestEditor tool or command test
A real browser or Electron contract (DOM events, focus, native menus, dialogs, windows, processes, files)	Semantic Playwright test
Appearance: colours, strokes, handles, layout	One focused golden, after a semantic precondition
Hardware rendering, GPU residency, WebGPU presentation	gpu project test
Save, quit, crash, recovery, file activation, cross-OS paths	platform project test

If most assertions in a draft E2E test read editor state after page.evaluate, the behavior probably belongs in TestEditor. Keep the E2E test as a thin proof that the real UI reaches it.

Waiting: never sleep

waitForTimeout is banned. Wait for the condition that makes the next step valid:

Before you…	Wait for
Read geometry, project coordinates, or send pointer input after setup changed geometry or camera	editor.waitForCanvasRender()
Assert after raw page.mouse moves	editor.flushPointerMoves(), then waitForCanvasRender() or waitForIdle()
Act on an editor route	waitForEditorReady() / openCatalogGlyph() (works for authored and preview sessions)
Click a catalog cell by coordinates	clickFirstCatalogGlyph() — it waits for a laid-out, settled Grid
Act on a workspace	waitForWorkspaceReady()
Assert after quit or close	dirtyDocumentDecisions() / dirtyDocumentRequests() — main-process records, not elapsed time
Compare Grid frames after an edit	Published Grid state (data-grid-readiness, data-atlas-build-count, active location), armed after setup settles

When elapsed time is itself the contract (debounce, momentum, idle timeouts), put the rule in a pure function driven by timestamps and unit-test it. If the E2E test must remain, dispatch events inside the page and wait on the page clock (performance.now()) that stamps them. glyph-view.spec.ts zoom momentum is the template.

Negative assertions ("nothing else happened") need a positive anchor: wait for the event that would have triggered the unwanted outcome, then assert the outcome is absent.

Oracles: assert what the editor decided, not what pixels look like

Never count or sample hard-coded colours from a canvas. Colour oracles break under themes, alpha, anti-aliasing, device scale, and renderer changes, and many pass when the feature is broken (an empty canvas has zero "wrong" pixels).

Use published state instead:

• geometry: editor.outline(), pointPosition(), selectionIds(), selectionBounds();
• interaction: toolState(), hoverId(), activeSnapGuides();
• rendering decisions: visibleOutlines(nodeId) and handleStates(node) on the glyph node definition;
• UI state: roles, accessible names, aria-pressed, aria-checked, aria-disabled, data-* state published for tests;
• persistence: savedGlyphNames() for .shift files and exportedGlyphNames() for exported fonts — never existsSync, byte inequality, or size > 0.

If the state you need is not published, publish it from the production code that renders it (same computation, not a parallel reimplementation), add a unit test for it, then assert it. That is how visibleOutlines and handleStates were added.

Assertions must be exact enough to fail on the wrong answer:

• assert the exact set or order, not count() > N or not.toEqual(before);
• derive expectations from the fixture or the UI (for example, read the ordered category buttons and compute the expected range) instead of hard-coding one neighbour;
• run the mental-deletion check: if the feature were a no-op, would this assertion fail?

Goldens: one contract, one focused capture

All goldens go through fixtures/snapshots.ts; scripts/check-e2e-projects.mjs rejects toHaveScreenshot or toMatchSnapshot anywhere else.

• expectCanvasSnapshot(editor, name) — the editor canvas stack, exact comparison, after the canvas has rendered.
• expectPanelSnapshot(locator, name) — one panel, menu, toolbar, or dialog.
• expectPageSnapshot(page, name) — full window; use sparingly, only for overall chrome.

Before every golden:

1. Assert the semantic precondition that makes the image meaningful: the point count, selection, tool state, outline targets, handle states, zoom, active theme (data-color-theme), or devicePixelRatio.
2. Park the pointer off the captured element unless hover is the contract.
3. Capture the smallest element that shows the contract. Do not use a full-page golden to protect a toolbar button.

Rules:

• One golden per distinct visual contract. Do not add near-duplicate images; if two captures are byte-identical, one of them is redundant.
• Canvas goldens use maxDiffPixels: 0 with threshold: 0.02 on every host: enough for antialiasing differences between hosts (≤ 0.009), far below a real colour-token change (~0.07). Never raise either to make a mismatch pass; the default threshold of 0.2 silently accepts token changes.
• Interface goldens (expectPanelSnapshot, expectPageSnapshot) are exact but compared on CI only, with baselines generated on the runner via the ci: update visual snapshots label. Never commit a locally generated interface baseline. Prefer semantic assertions and keep interface goldens few.
• Tag every test that captures a golden: test(title, { tag: "@golden" }, …). The snapshot label regenerates only @golden tests, and the helpers fail an untagged golden.
• Goldens never pass on retry. The helpers throw on a retry attempt, so explain the first failure.
• Use canvas-local positions (dragCanvas(), canvasPagePoint()) so layout changes cannot redraw geometry.
• Theme goldens select the theme through the product and assert data-color-theme first. HiDPI goldens use test.use({ deviceScaleFactor: 2 }) and assert devicePixelRatio first. Add one golden per palette branch or rendering path, not one per theme.

Updating baselines: only for an intentional appearance change. Run the focused spec with --update-snapshots -g "<title>", open every changed PNG and describe what changed, then rerun without update mode and with --retries=0 --repeat-each=3. Canvas baselines may be generated locally. Interface baselines are generated only by the ci: update visual snapshots label; inspect the runner-generated images before merging.

Fixtures and isolation

• Use the shared editor fixture; construct EditorDriver directly only for additional windows.
• Relaunch through the relaunch fixture, never raw electron.launch. The process registry owns every launched tree, attaches diagnostics, and kills it even when the test times out.
• Every launch environment comes from shiftTestEnvironment(). Never spread process.env into a launch.
• Script native dialogs with the fixture options (dirtyDocumentChoice(s), saveShiftPaths, openFontPath). An unexpected renderer dialog fails the test; opt in with allowRendererDialogs only when the dialog is the behavior.
• Put shared setup in fixtures or EditorDriver (openScratchGlyph(), commitInputValue(), toolButton()), not copied helpers in specs. Do not grow EditorDriver with one-off helpers.
• Prefer getByRole() / getByLabel() and locators from fixtures/appLocators.ts. No parent traversal (locator("..")), no styling classes, no toHaveCSS() unless the style value is the product contract.

Structure

• One behavior per test. Split long flows when an early failure would hide unrelated coverage.
• Do not accumulate state across unrelated steps; start each test from a fixture.
• Setup may use page.evaluate or insertContent; the behavior under test must go through the user surface.
• Name tests after the behavior: "shift-selects glyph categories as a visible range", not "category test".

Projects

Membership is explicit in apps/desktop/playwright.config.ts. Add a new spec to the right list — visual, platform, gpu, or perf — and run node scripts/check-e2e-projects.mjs. A spec that needs native lifecycle behavior belongs in platform, which runs fully on Linux. Tag a platform test @os only when its behavior differs by OS (window controls, signals, native menus, file activation and paths); only @os tests run on macOS and Windows. A spec whose only GPU dependency is incidental belongs in visual.

Large-data performance regressions

• The perf project runs nightly or manually, not in the normal correctness suite. Protect a large-list UI change with a small visual interaction test that reaches an offscreen row, preserves selection and keyboard focus, and bounds the number of mounted rows.
• In a real-app performance test, time the whole user-visible operation through confirmed edit settlement and canvas readiness with the relevant panels mounted. Timing only a local mutation, Rust apply, or IPC handler misses renderer main-thread stalls. Keep hardware-sensitive time ceilings in perf, not visual; verify the test fails for the original regression rather than only when it times out.
• Use EditorDriver for navigation and edit settling. Keep the performance fixture responsible for the hardware launch and generated data, not editor behavior.

Proving a test is not flaky

Before committing a new or changed E2E test:

pnpm test:e2e:visual e2e/<spec>.spec.ts -g "<title>" --repeat-each=10 --retries=0

For timing-sensitive flows, repeat under CPU load (for example, several yes > /dev/null & processes, killed afterwards), because CI runners are slower than development machines. For platform behavior, rely on the Windows and Linux CI jobs and say so in the pull request.

When a test fails or flakes:

1. Read the trace and electron-diagnostics before changing anything.
2. Classify it: product regression, test oracle, or infrastructure. Do not call a single failure flaky without a retry pass or a rerun on the same commit.
3. Fix the oracle or the wait. Never add a sleep, a retry, or tolerance, and never regenerate a baseline without understanding the diff.

The CI "E2E Report" job lists tests that passed only on retry. Treat every entry as a bug to fix, not noise.

Review checklist

• The layer is right; logic that TestEditor can observe is tested there.
• No waitForTimeout, no fixed delays, and no polling without a semantic condition.
• No colour counting, pixel sampling, byte inequality, or "something changed" oracle.
• Assertions are exact and fail when the feature is a no-op.
• Every golden goes through fixtures/snapshots.ts, captures one contract at the smallest element, and follows a semantic precondition.
• Changed baselines were inspected and verified with --retries=0.
• Launches, relaunches, and dialogs go through fixtures.
• The spec belongs to the intended project, and the project check passes.
• The test passed --repeat-each=10 --retries=0, and the pull request lists the exact E2E commands and any coverage not run.