Test Vision

workflow no always-loaded rules -- loads on demand updated 2026-08-04

Vision-driven e2e test suite generation. /test-vision for full repo analysis + parallel test suite creation. /e2e for single-feature spec generation. Composes /e2e as the atomic unit within /test-vision orchestration.

README

Test Vision Module

Vision-driven e2e test suite generation for any codebase. Provides two composable commands:

/test-vision - Full repo analysis: discovers all features, interviews you to validate test cases, generates Playwright infrastructure, dispatches parallel agents to build a complete e2e test suite with CI/CD integration.
/e2e - Single feature: generates one Playwright spec file for a specific feature, flow, or issue. Works standalone or as the atomic building block within /test-vision.

Architecture

/test-vision composes /e2e as its atomic unit. The orchestrator handles discovery, infrastructure, and coordination. Each feature domain gets its own /e2e agent that generates a single spec file. File paths are pre-assigned before dispatch to prevent conflicts.

/test-vision
  |-- Phase 0: Codebase Discovery (7-source checklist)
  |-- Phase 1: Chrome MCP Visual Discovery
  |-- Phase 2: User Interview (validate, prioritize, sign off)
  |-- Phase 3: Infrastructure Generation (config, fixtures, auth)
  |-- Phase 4: Parallel /e2e Dispatch (one agent per feature domain)
  |-- Phase 5: Integration & Validation
  |-- Phase 6: CI/CD Workflow Generation
  |-- Phase 7: Report

Usage

Full Test Suite Generation

# Run in any repo to generate a complete e2e test suite
/test-vision

# Skip Chrome MCP discovery (code-based only)
/test-vision --skip-chrome

# Skip user interview (use auto-detected defaults)
/test-vision --skip-interview

Single Feature Spec

# By feature name
/e2e authentication

# By issue number
/e2e #42

# By route path
/e2e /dashboard/settings

# By description
/e2e user profile editing with avatar upload

# With explicit output path
/e2e payments --file e2e/features/payments.spec.ts

What Gets Generated

Test Infrastructure

playwright.config.ts - with auth setup project, webServer config
e2e/fixtures.ts - authenticatedPage fixture with graceful skip
e2e/auth.setup.ts - auth provider-specific setup (Better Auth, Supabase, Clerk)
e2e/.auth/.gitignore - ignore auth state files

Spec Files

e2e/features/{domain}.spec.ts - one per feature domain
Three-tier assertions: route loads, structural landmarks, behavioral interactions
Direct locators (getByRole, getByText, getByTestId)
Graceful credential skipping

CI/CD

.github/workflows/e2e.yml - GitHub Actions workflow with artifact upload

Test-Driven Feature Development

The generated test suite supports building features test-first:

Run /test-vision to generate specs for all planned features
Tests for unbuilt features will fail (this is expected)
Give an agent the failing spec: "Using the e2e tests in e2e/features/payments.spec.ts and Playwright, build out the payments feature"
The agent builds the feature to make the tests pass
Use /e2e to add specs for new features as they're designed

Supported Auth Providers

Provider	Strategy	Env Vars
Better Auth	UI-based form fill	E2E_USER_EMAIL, E2E_USER_PASSWORD
Supabase	Programmatic API injection	E2E_USER_EMAIL, E2E_USER_PASSWORD, VITE_SUPABASE_URL, VITE_SUPABASE_PUBLISHABLE_KEY
Clerk	UI-based two-step sign-in	E2E_USER_EMAIL, E2E_USER_PASSWORD
Unknown	Generic UI-based (customizable)	E2E_USER_EMAIL, E2E_USER_PASSWORD

Dependencies

browser-automation - Chrome MCP tool permissions for visual discovery
multi-agent - agent dispatch for parallel spec generation

Manual Installation

mkdir -p ~/.claude/commands
cp commands/test-vision.md ~/.claude/commands/test-vision.md
cp commands/e2e.md ~/.claude/commands/e2e.md

Then ensure the browser-automation and multi-agent modules are installed — both are required at runtime.

Will install

Path	Action	Target	Type
`commands/test-vision.md`	→	`commands/test-vision.md`	command
`commands/e2e.md`	→	`commands/e2e.md`	command

Dependencies

Required by

No other module depends on this one.

Included in presets

Install this module

Agent prompt

Recommended for agent users -- hands the whole install off to your assistant.

Fetch https://cd23a9be.ccgm-site.pages.dev/modules/test-vision.md and install this module into my Claude Code setup.

Native plugin marketplace

One command via the native plugin marketplace -- additive, does not merge settings.json.

claude plugin install test-vision@ccgm

The marketplace path is additive, not a replacement: it installs commands, agents, and skills as native plugin components, but it does not perform the bash installer's deep settings.json merge, and it does not write the always-loaded global CLAUDE.md context. Rules are only injected via an opt-in SessionStart hook rather than being auto-loaded. Use the bash installer when those pieces matter to you.

Manual, per file

Full control -- copy exactly the files you want from the sections below.

Files

command (2)

commands/test-vision.md

---
description: Comprehensive e2e test suite generation. Discovers all features, interviews user, generates infrastructure, dispatches parallel /e2e agents.
allowed-tools: Bash, Read, Write, Edit, Glob, Grep, Agent, AskUserQuestion
argument-hint: "[--skip-chrome] [--skip-interview]"
---

# /test-vision - Vision-Driven E2E Test Suite Generation

Discovers all features in a codebase, interviews the user to validate test cases and priorities, generates shared Playwright infrastructure, then dispatches parallel agents to build a complete e2e test suite.

Composes `/e2e` as its atomic unit: each feature domain gets its own `/e2e` agent that generates a single spec file.

**Flags:**
- `--skip-chrome` - Skip Chrome MCP visual discovery (use code-based discovery only)
- `--skip-interview` - Skip user interview, use auto-detected defaults (prints what was detected)

## Sub-Agent Model Optimization

| Phase | Agent | Model |
|-------|-------|-------|
| Phase 0 | Discovery agent | sonnet |
| Phase 4 | Spec generation agents (/e2e) | sonnet |

The orchestrator (this session) stays on the current model for synthesis, interview, and infrastructure generation.

---

## Input

```
$ARGUMENTS
```

---

## Phase 0: Codebase Discovery

Launch a single Agent (model: sonnet) with the following task:

```
Analyze this codebase and produce a feature-domain map for e2e test planning.

## Pre-check: Monorepo Detection

First, check if this is a monorepo:
- ls pnpm-workspace.yaml turbo.json nx.json 2>/dev/null
- cat package.json | grep '"workspaces"' 2>/dev/null

If a monorepo is detected, list the apps/packages and STOP. Report back which apps exist so the orchestrator can ask the user which to target. Do NOT proceed with discovery in a monorepo until an app is selected.

## 7-Source Discovery Checklist

Run all of these:

1. **Route definitions**:
   - Glob: **/routes.{ts,tsx}, **/App.{ts,tsx}, **/router.{ts,tsx}, **/app/**/page.{ts,tsx}
   - Extract: path, component, auth requirement (look for ProtectedRoute, RequireAuth, or similar wrappers)

2. **Navigation component**:
   - Grep for: <nav, <Sidebar, <NavLink, <Link to=, <Link href=
   - Extract: visible navigation links and their destinations

3. **README + CLAUDE.md**:
   - Read both files if they exist
   - Extract: feature descriptions, user flow descriptions, route tables

4. **API endpoints**:
   - Glob: **/api/**/*.{ts,js}, **/functions/**/*.{ts,js}, **/routes/**/*.{ts,js}
   - Exclude node_modules
   - Extract: endpoint paths, HTTP methods, what they do

5. **Existing test coverage**:
   - Glob: **/*.spec.ts, **/*.test.ts, **/e2e/**
   - Extract: which features are already tested, which are not

6. **State stores**:
   - Glob: **/store/**/*.{ts,tsx}, **/stores/**/*.{ts,tsx}, **/context/**/*.{ts,tsx}
   - Extract: store names, actions, state shape (reveals features without dedicated routes)

7. **Form schemas**:
   - Grep for: z.object, useForm, zodResolver, yupResolver
   - Extract: form fields, validation rules, error states

## Auth Provider Detection

Check package.json dependencies for:
- better-auth -> "Better Auth"
- @supabase/supabase-js or @supabase/ssr -> "Supabase"
- @clerk/clerk-react or @clerk/nextjs -> "Clerk"
- None -> "Unknown / No Auth"

## Output Format

Return a structured feature-domain map:

Feature Domain Map:
  1. {Feature Name} (Tier {0|1|2})
     Routes: {/path1, /path2}
     Components: {ComponentA, ComponentB}
     API: {GET /api/path, POST /api/path}
     State: {storeName (if applicable)}
     Forms: {formName with N fields (if applicable)}
     Existing tests: {none | partial | full}

  2. ...

Auth Provider: {detected provider}
Total Feature Domains: {N}
```

### Monorepo Handling

If the discovery agent reports a monorepo, use AskUserQuestion:

```
"This is a monorepo with multiple apps: {list}. Which app should I generate e2e tests for?"
```

Options: one per app detected. After selection, re-run discovery scoped to that app's directory.

---

## Phase 1: Chrome MCP Visual Discovery

**Skip this phase if `--skip-chrome` flag is set or Chrome MCP tools are unavailable.**

For each route in the feature-domain map:

1. Get browser context: `tabs_context_mcp(createIfEmpty: true)`
2. Navigate to the route
3. If redirected to a login/auth page: mark as "auth-required, Chrome skipped" and continue to next route
4. If loads successfully:
   - Read the page content (`read_page`) to identify interactive elements
   - Note: page title, headings, form fields, buttons, CTAs, modals, data displays
   - Check console for JS errors (`read_console_messages`)
   - Check network for API call patterns (`read_network_requests`)

5. Enrich the feature-domain map with visual context:
   - Actual page titles and headings found
   - Form field labels and types
   - Button text and available actions
   - Data display patterns (tables, lists, cards)
   - Any JS errors or failed network requests observed

Partial data is expected. Public routes get full Chrome context; auth-required routes rely on code-based discovery only.

---

## Phase 2: User Interview

**Skip this phase if `--skip-interview` flag is set. Instead, print auto-detected defaults:**
```
--skip-interview mode:
  Auth provider: {detected}
  Feature domains: {N}
  Auth credentials expected: E2E_USER_EMAIL, E2E_USER_PASSWORD
  Generating with defaults... (run without --skip-interview to customize)
```

### Step 1: Feature Validation

Present the feature-domain map using AskUserQuestion:

```
question: "I've identified {N} feature domains in this codebase:

{Feature-domain map summary - show feature names, route counts, auth tiers}

Does this cover everything?"
options:
  - "Looks complete - proceed"
  - "Missing features (I'll describe)"
  - "Remove some (I'll specify)"
```

For repos with >12 domains, show a condensed summary (feature name + route count only) rather than the full map. Expand on request.

### Step 1b: Unbuilt Features

```
question: "Are there any planned features not yet in the codebase that you'd like test specs for? These specs will use permissive assertions so they serve as executable specifications for agents building the features."
options:
  - "No, just test what exists"
  - "Yes (I'll describe the planned features)"
```

If yes, add the described features to the domain map with a `[PLANNED]` marker.

### Step 2: Priority & Scope

```
question: "Which features are highest priority for e2e coverage?"
options:
  - "All of them - full coverage"
  - "Critical path only (I'll specify which)"
  - "Let me rank them"
```

### Step 3: Auth Setup

```
question: "I detected {auth provider}. For e2e tests, I'll generate auth.setup.ts using {strategy description}. Test credentials will be read from env vars (E2E_USER_EMAIL, E2E_USER_PASSWORD). Sound right?"
options:
  - "Yes, proceed"
  - "Different auth approach (I'll describe)"
```

### Step 4: Delegation Review

```
question: "Here's how I'll delegate the work:

{N} parallel agents, each generating one spec file:
  Agent 1 -> e2e/features/{domain1}.spec.ts (~{M} tests)
  Agent 2 -> e2e/features/{domain2}.spec.ts (~{M} tests)
  ...

Estimated total: {N} spec files, ~{M} total tests.

Ready to proceed?"
options:
  - "Proceed with spec generation"
  - "Adjust delegation (I'll describe changes)"
```

---

## Phase 3: Infrastructure Generation

Generate the shared test infrastructure using the Write tool. All files must be written and verified before Phase 4.

### 3.1 Install Playwright (if needed)

```bash
grep -q "@playwright/test" package.json 2>/dev/null || npm install -D @playwright/test
npx playwright install chromium 2>/dev/null
```

### 3.2 Generate playwright.config.ts

If `playwright.config.ts` does not already exist, generate it.

**With auth detected:**
```typescript
import { defineConfig, devices } from '@playwright/test'

export default defineConfig({
  testDir: './e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 2 : undefined,
  reporter: 'html',
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:5173',
    trace: 'on-first-retry',
  },
  projects: [
    { name: 'setup', testMatch: /auth\.setup\.ts/ },
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
      dependencies: ['setup'],
    },
  ],
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:5173',
    reuseExistingServer: !process.env.CI,
    timeout: 120000,
  },
})
```

**Without auth:**
Omit the `setup` project and `dependencies`.

### 3.3 Generate e2e/fixtures.ts (if auth detected)

```typescript
import { test as base, type Page } from '@playwright/test'
import { existsSync } from 'fs'

const USER_STORAGE_STATE = 'e2e/.auth/user.json'

export const test = base.extend<{
  authenticatedPage: Page
}>({
  authenticatedPage: async ({ browser }, use, testInfo) => {
    if (!existsSync(USER_STORAGE_STATE)) {
      testInfo.skip(true, 'Auth not configured. Set E2E_USER_EMAIL and E2E_USER_PASSWORD env vars, then run: npx playwright test --project=setup')
      return
    }
    const context = await browser.newContext({ storageState: USER_STORAGE_STATE })
    const page = await context.newPage()
    await use(page)
    await context.close()
  },
})

export { expect } from '@playwright/test'
```

### 3.4 Generate e2e/auth.setup.ts (if auth detected)

Select the appropriate template based on detected auth provider:

- **Better Auth**: UI-based email/password form fill
- **Supabase**: Programmatic API with `signInWithPassword()` and session injection
- **Clerk**: UI-based with Clerk's two-step sign-in flow
- **Unknown**: Generic UI-based with customization comments

See `/e2e` command for the full templates.

### 3.5 Create Directories and Gitignore

```bash
mkdir -p e2e/features e2e/.auth
echo '*' > e2e/.auth/.gitignore
grep -q "e2e/.auth" .gitignore 2>/dev/null || echo "e2e/.auth/" >> .gitignore
```

### 3.6 Verify Infrastructure (MANDATORY)

Before proceeding to Phase 4, verify ALL infrastructure files exist:

```bash
ls -la playwright.config.ts e2e/fixtures.ts e2e/auth.setup.ts e2e/features/ 2>&1
```

If auth was not detected, `e2e/fixtures.ts` and `e2e/auth.setup.ts` may not exist - that is correct. But `playwright.config.ts` and `e2e/features/` must always exist.

**Do NOT proceed to Phase 4 until this verification passes.**

---

## Phase 4: Parallel /e2e Dispatch

### 4.1 Pre-Assign File Paths

For each feature domain, assign a unique output file:
```
{domain-1} -> e2e/features/{domain-1-kebab}.spec.ts
{domain-2} -> e2e/features/{domain-2-kebab}.spec.ts
...
```

No two agents may touch the same file.

### 4.2 Scalability

For repos with >12 feature domains, split into two dispatch waves of 6-8 agents each. This keeps orchestrator context manageable. Files don't conflict between waves, so this is purely for orchestrator management.

> **Concurrency — avoid the 429 throttle.** The wave cap above is also what keeps the fan-out under the server-side rate limit. The `/e2e` agents are light (`model: sonnet`), so 6–8 per wave is fine; never launch all domains for a large repo in one burst, and if you escalate the agents to a heavier model, drop the wave size to ≤4. If a wave returns `Server is temporarily limiting requests · Rate limited`, wait 30–60s and re-dispatch the failed domains. See `~/.claude/rules/concurrency-and-rate-limits.md`.

### 4.3 Dispatch Agents

For each feature domain, spawn an agent (model: sonnet) with the following prompt:

```
Read the file ~/.claude/commands/e2e.md and follow its instructions exactly.

CONTEXT (from /test-vision discovery):
- Feature: {feature name}
- Auth tier: {0=public, 1=authenticated, 2=admin}
- Routes: {list of routes for this feature}
- Components: {key components identified}
- API endpoints: {related API paths}
- Interactive elements: {forms, buttons, modals from Chrome MCP if available}
- Visual context: {page titles, headings, CTA text from Chrome MCP if available}
- Auth provider: {detected provider}
- Planned/unbuilt: {yes/no - if yes, use permissive three-tier assertions}

CONSTRAINTS:
- Output file: {pre-assigned path}
- Import from '../fixtures' for authenticatedPage (or '@playwright/test' if no auth)
- Use direct locators (getByRole, getByText, getByTestId, getByLabel)
- Every test calls page.goto() directly - no shared navigation state
- Graceful credential skipping via the fixture
- Use .or() for features with multiple valid states (empty vs populated)
- No exact copy assertions - use regex matchers
- Aim for 6-15 tests
- Three-tier assertions: route loads, structural landmarks, behavioral interactions

Write the complete spec file to the output path.
```

### 4.4 Wait for Completion

Wait for all dispatched agents to complete. Track which succeeded and which failed.

---

## Phase 5: Integration & Validation

After all /e2e agents complete:

### 5.1 Verify Files Created

```bash
ls -la e2e/features/*.spec.ts
```

Compare against the pre-assigned file list. Report any missing files.

### 5.2 Run Test Discovery

```bash
npx playwright test --list 2>&1
```

All generated tests should be discoverable. If any file has syntax errors, fix them.

### 5.3 Check for Import Path Errors

```bash
grep -rn "from.*fixtures" e2e/features/ | grep -v "../fixtures"
```

Any result here is a misconfigured import path. Fix `./fixtures` to `../fixtures`.

### 5.4 Check for Duplicate Test Names

```bash
grep -rh "test('" e2e/features/ | sort | uniq -d
```

If duplicates exist, rename them to be unique (add the feature name as prefix).

### 5.5 Smoke Check (optional)

If a dev server is running, run the full suite:

```bash
npx playwright test --project=chromium 2>&1 | tail -30
```

Report results. Tests for unbuilt features are expected to fail - this is correct behavior.

---

## Phase 6: CI/CD Workflow Generation

Generate `.github/workflows/e2e.yml` (or update existing):

```bash
mkdir -p .github/workflows
```

```yaml
name: E2E Tests

on:
  pull_request:
    branches: [main]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test
        env:
          CI: true
          # Auth-dependent tests require these GitHub Actions secrets.
          # Set in: Settings > Secrets and variables > Actions
          # Tests gracefully skip if not configured.
          E2E_USER_EMAIL: ${{ secrets.E2E_USER_EMAIL }}
          E2E_USER_PASSWORD: ${{ secrets.E2E_USER_PASSWORD }}
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30
```

---

## Phase 7: Report

Present the final summary:

```
Test Vision Complete

Feature Domains: {N}
Spec Files Generated: {N}
Total Tests: {N}
Infrastructure Files: {N} (playwright.config.ts, fixtures.ts, auth.setup.ts)
CI Workflow: .github/workflows/e2e.yml

Coverage by Domain:
  {domain}.spec.ts - {N} tests (Tier {0|1|2})
  {domain}.spec.ts - {N} tests (Tier {0|1|2})
  ...

{If any planned/unbuilt features:}
Tests for Planned Features (expected to fail until built):
  {feature name} - {N} tests in {file}

Validation:
  Test discovery: {pass/fail}
  Import paths: {pass/fail}
  Duplicate names: {pass/fail}
  Smoke check: {pass/fail/skipped}

Next Steps:
  1. Set E2E_USER_EMAIL and E2E_USER_PASSWORD in .env
  2. Run: npx playwright test --project=setup (authenticate)
  3. Run: npx playwright test (run full suite)
  4. Add GitHub Actions secrets for CI (Settings > Secrets)
  5. Use /e2e to add tests for new features as they're built
```

commands/e2e.md

---
description: Generate a Playwright e2e test spec for a single feature, flow, or issue. Works standalone or as a building block for /test-vision.
allowed-tools: Bash, Read, Write, Edit, Glob, Grep, Agent, AskUserQuestion
argument-hint: <feature name | issue number | route path | description> [--file <output-path>]
---

# /e2e - Playwright E2E Spec Generator

Generates a complete Playwright e2e test spec file for a single feature, flow, or issue. Works in two modes:

- **Standalone**: Called directly by the user. Runs full discovery + Chrome MCP exploration + infrastructure setup + spec generation.
- **Composed**: Called by /test-vision as an atomic building block. Receives pre-computed context and skips discovery.

## Sub-Agent Model Optimization

When spawning discovery sub-agents, use **sonnet**. The orchestrator (this command) stays on the current model.

---

## Input

```
$ARGUMENTS
```

---

## Phase 0: Mode Detection (READ THIS FIRST)

**This is a hard branch. Follow it exactly.**

Check the agent prompt/context for a block beginning with `CONTEXT (from /test-vision discovery):`.

- **If that block IS present**: You are in **COMPOSED MODE**.
  - Skip Phase 1 (Parse & Discover) entirely.
  - Skip Phase 2 (Chrome MCP Exploration) entirely.
  - Phase 3 becomes **check-only** (verify infrastructure exists, NEVER write files).
  - Proceed directly to Phase 4 (Generate Spec) using the provided context.

- **If that block is NOT present**: You are in **STANDALONE MODE**.
  - Run all phases starting from Phase 1.

Do NOT run discovery if composed context is provided. Do NOT run Chrome MCP if composed context is provided. This is non-negotiable.

---

## Phase 1: Parse & Discover (STANDALONE MODE ONLY)

### 1.1 Parse Arguments

Parse `$ARGUMENTS`:
- If starts with `#` or is a number: treat as GitHub issue number
  ```bash
  gh issue view {number} --json title,body,labels
  ```
- If starts with `/`: treat as a route path, search for the feature owning that route
- Otherwise: treat as a feature name or description

### 1.2 Targeted Discovery

Run a focused discovery for the specific feature area:

1. **Find route definitions** matching the feature:
   ```
   Grep for the route path or feature name in: **/routes.{ts,tsx}, **/App.{ts,tsx}, **/router.{ts,tsx}
   ```

2. **Read relevant components**:
   ```
   Glob for components related to the feature in src/
   ```

3. **Check for existing tests**:
   ```
   Glob: e2e/**/*.spec.ts, **/{feature}*.spec.ts, **/{feature}*.test.ts
   ```

4. **Identify auth requirements**:
   - Look for ProtectedRoute wrappers, auth middleware, or auth checks on the routes
   - Check for `useAuth`, `useSession`, or similar hooks in the components

5. **Find related API endpoints**:
   ```
   Grep for API routes related to the feature: **/api/**/{feature}*, **/functions/**/{feature}*
   ```

6. **Check state stores** (if applicable):
   ```
   Glob: **/store/**/{feature}*, **/stores/**/{feature}*, **/context/**/{feature}*
   ```

7. **Find form schemas**:
   ```
   Grep for zod schemas or useForm in the feature's components
   ```

### 1.3 Detect Auth Provider

Check `package.json` for the auth provider:
- `better-auth` -> Better Auth
- `@supabase/supabase-js` or `@supabase/ssr` -> Supabase Auth
- `@clerk/clerk-react` or `@clerk/nextjs` -> Clerk
- None detected -> no auth or unknown provider

---

## Phase 2: Chrome MCP Exploration (STANDALONE MODE ONLY)

**Skip if Chrome MCP tools are unavailable or the dev server is not running.**

1. Check if a dev server is running:
   ```bash
   curl -s -o /dev/null -w "%{http_code}" http://localhost:5173 2>/dev/null || \
   curl -s -o /dev/null -w "%{http_code}" http://localhost:3000 2>/dev/null
   ```

2. If the app is running and Chrome MCP tools are available:
   - Navigate to the feature's primary route
   - Read the page content to identify interactive elements
   - Note: forms, buttons, modals, dropdowns, data displays
   - Check console for errors
   - If redirected to login/auth: note as "auth-required" and proceed with code-based discovery only

3. If Chrome MCP unavailable or app not running: skip this phase and note in output. Code-based discovery from Phase 1 is sufficient.

---

## Phase 3: Infrastructure Check/Generate

### Standalone Mode

Before writing the spec, ensure infrastructure exists. For each missing file, generate it:

1. **Check for Playwright**:
   ```bash
   grep -q "@playwright/test" package.json 2>/dev/null
   ```
   If missing:
   ```bash
   npm install -D @playwright/test
   npx playwright install chromium
   ```

2. **Check for `playwright.config.ts`**:
   If missing, generate based on auth detection:

   **With auth detected:**
   ```typescript
   import { defineConfig, devices } from '@playwright/test'

   export default defineConfig({
     testDir: './e2e',
     fullyParallel: true,
     forbidOnly: !!process.env.CI,
     retries: process.env.CI ? 2 : 0,
     workers: process.env.CI ? 2 : undefined,
     reporter: 'html',
     use: {
       baseURL: process.env.BASE_URL || 'http://localhost:5173',
       trace: 'on-first-retry',
     },
     projects: [
       { name: 'setup', testMatch: /auth\.setup\.ts/ },
       {
         name: 'chromium',
         use: { ...devices['Desktop Chrome'] },
         dependencies: ['setup'],
       },
     ],
     webServer: {
       command: 'npm run dev',
       url: 'http://localhost:5173',
       reuseExistingServer: !process.env.CI,
       timeout: 120000,
     },
   })
   ```

   **Without auth:**
   ```typescript
   import { defineConfig, devices } from '@playwright/test'

   export default defineConfig({
     testDir: './e2e',
     fullyParallel: true,
     forbidOnly: !!process.env.CI,
     retries: process.env.CI ? 2 : 0,
     workers: process.env.CI ? 2 : undefined,
     reporter: 'html',
     use: {
       baseURL: process.env.BASE_URL || 'http://localhost:5173',
       trace: 'on-first-retry',
     },
     projects: [
       {
         name: 'chromium',
         use: { ...devices['Desktop Chrome'] },
       },
     ],
     webServer: {
       command: 'npm run dev',
       url: 'http://localhost:5173',
       reuseExistingServer: !process.env.CI,
       timeout: 120000,
     },
   })
   ```

3. **Check for `e2e/fixtures.ts`** (only if auth detected):
   If missing, generate:
   ```typescript
   import { test as base, type Page } from '@playwright/test'
   import { existsSync } from 'fs'

   const USER_STORAGE_STATE = 'e2e/.auth/user.json'

   export const test = base.extend<{
     authenticatedPage: Page
   }>({
     authenticatedPage: async ({ browser }, use, testInfo) => {
       if (!existsSync(USER_STORAGE_STATE)) {
         testInfo.skip(true, 'Auth not configured. Set E2E_USER_EMAIL and E2E_USER_PASSWORD env vars, then run: npx playwright test --project=setup')
         return
       }
       const context = await browser.newContext({ storageState: USER_STORAGE_STATE })
       const page = await context.newPage()
       await use(page)
       await context.close()
     },
   })

   export { expect } from '@playwright/test'
   ```

4. **Check for `e2e/auth.setup.ts`** (only if auth detected):
   If missing, generate based on provider. See auth templates below.

5. **Create directories**:
   ```bash
   mkdir -p e2e/features e2e/.auth
   ```

6. **Gitignore auth state**:
   ```bash
   echo '*' > e2e/.auth/.gitignore
   grep -q "e2e/.auth" .gitignore 2>/dev/null || echo "e2e/.auth/" >> .gitignore
   ```

### Composed Mode (CHECK-ONLY)

Verify all infrastructure files exist:
```bash
ls e2e/fixtures.ts e2e/auth.setup.ts playwright.config.ts 2>/dev/null
ls e2e/features/ 2>/dev/null
```

If any required file is missing, report the error: "Infrastructure file missing: {file}. This should have been generated by /test-vision Phase 3. Aborting." Do NOT attempt to create infrastructure files in composed mode.

---

## Phase 4: Generate Spec File

Determine the output file path:
- If `--file <path>` was provided: use that path
- If in composed mode: use the path from the context block
- Otherwise: default to `e2e/features/{feature-name-kebab}.spec.ts`

Generate the spec file following this template:

```typescript
import { test, expect } from '../fixtures'
// For projects without auth, use: import { test, expect } from '@playwright/test'

/**
 * {Feature Name} E2E Tests
 *
 * Covers: {list the flows covered}
 * Auth required: {yes/no}
 * Routes tested: {/path1, /path2}
 * Generated by: /e2e
 */

test.describe('{Feature Name}', () => {

  // ---- Tier 1: Route loads ----

  test.describe('Page Load', () => {
    test('{feature} page loads without errors', async ({ page }) => {
      await page.goto('{route}')
      await expect(page).not.toHaveURL(/error|404/)
      await expect(page.getByRole('main')).toBeVisible()
    })
  })

  // ---- Tier 2: Structural landmarks ----

  test.describe('Structure', () => {
    test('displays expected headings and navigation', async ({ authenticatedPage: page }) => {
      await page.goto('{route}')
      await expect(page.getByRole('heading', { name: /{feature}/i })).toBeVisible()
      // Add assertions for key UI landmarks
    })
  })

  // ---- Tier 3: Behavioral interactions ----

  test.describe('{Primary Flow}', () => {
    test('{expected behavior}', async ({ authenticatedPage: page }) => {
      await page.goto('{route}')
      await page.waitForLoadState('networkidle')
      // Add interaction assertions
    })
  })

  // ---- Error states ----

  test.describe('Error States', () => {
    test('redirects unauthenticated users', async ({ page }) => {
      await page.goto('{protected-route}')
      await expect(page).toHaveURL(/login|signin|sign-in/)
    })
  })

})
```

### Spec Generation Rules

- **Direct locators**: Use `getByRole`, `getByText`, `getByTestId`, `getByLabel`. No CSS selectors unless absolutely necessary.
- **No shared state**: Every test calls `page.goto()` directly. No state carried between tests.
- **Auth fixtures**: Protected routes use `authenticatedPage` fixture. Public routes use `page`.
- **Graceful skip**: Auth-dependent tests skip when credentials are absent (handled by the fixture).
- **`.or()` combinator**: For features with multiple valid states (empty vs populated), use `.or()` to accept any valid state.
- **No exact copy assertions**: Use regex matchers (`/pattern/i`) for text assertions. Copy changes shouldn't break tests.
- **Test count**: Aim for 6-15 tests per spec file. Fewer means missing coverage. More means testing implementation details.
- **Three tiers**: Always include Tier 1 (route loads). Include Tier 2 (structural) for all features with UI. Include Tier 3 (behavioral) for features with interactions.
- **Error states**: Include auth redirect tests for protected routes. Include form validation tests if forms exist.

### Auth Setup Templates (used in Phase 3)

#### Better Auth (email/password - UI-based)
```typescript
import { test as setup, expect } from '@playwright/test'

const USER_AUTH_FILE = 'e2e/.auth/user.json'

setup('authenticate as user', async ({ page }) => {
  const email = process.env.E2E_USER_EMAIL
  const password = process.env.E2E_USER_PASSWORD

  if (!email || !password) {
    console.log('Skipping auth setup: E2E_USER_EMAIL and E2E_USER_PASSWORD not set')
    return
  }

  await page.goto('/login')
  await page.getByLabel(/email/i).fill(email)
  await page.getByLabel(/password/i).fill(password)
  await page.getByRole('button', { name: /sign in|log in/i }).click()
  await page.waitForURL(/dashboard|home|\/$/)
  await page.context().storageState({ path: USER_AUTH_FILE })
})
```

#### Supabase Auth (programmatic API)
```typescript
import { test as setup } from '@playwright/test'
import { createClient } from '@supabase/supabase-js'

const USER_AUTH_FILE = 'e2e/.auth/user.json'

setup('authenticate via Supabase API', async ({ page }) => {
  const email = process.env.E2E_USER_EMAIL
  const password = process.env.E2E_USER_PASSWORD
  const supabaseUrl = process.env.VITE_SUPABASE_URL || process.env.SUPABASE_URL
  const supabaseKey = process.env.VITE_SUPABASE_PUBLISHABLE_KEY || process.env.SUPABASE_PUBLISHABLE_KEY

  if (!email || !password || !supabaseUrl || !supabaseKey) {
    console.log('Skipping auth setup: E2E credentials or Supabase config not set')
    return
  }

  const supabase = createClient(supabaseUrl, supabaseKey)
  const { data, error } = await supabase.auth.signInWithPassword({ email, password })

  if (error || !data.session) {
    console.log(`Auth failed: ${error?.message || 'no session'}`)
    return
  }

  await page.goto('/')
  await page.evaluate((session) => {
    const storageKey = Object.keys(localStorage).find(k => k.startsWith('sb-')) || 'sb-auth-token'
    localStorage.setItem(storageKey, JSON.stringify(session))
  }, data.session)
  await page.reload()
  await page.waitForURL(/dashboard|home|\/$/)
  await page.context().storageState({ path: USER_AUTH_FILE })
})
```

#### Clerk (UI-based)
```typescript
import { test as setup } from '@playwright/test'

const USER_AUTH_FILE = 'e2e/.auth/user.json'

setup('authenticate as user', async ({ page }) => {
  const email = process.env.E2E_USER_EMAIL
  const password = process.env.E2E_USER_PASSWORD

  if (!email || !password) {
    console.log('Skipping auth setup: E2E_USER_EMAIL and E2E_USER_PASSWORD not set')
    return
  }

  await page.goto('/sign-in')
  await page.getByLabel(/email/i).fill(email)
  await page.getByRole('button', { name: /continue/i }).click()
  await page.getByLabel(/password/i).fill(password)
  await page.getByRole('button', { name: /continue|sign in/i }).click()
  await page.waitForURL(/dashboard|home|\/$/)
  await page.context().storageState({ path: USER_AUTH_FILE })
  // If Clerk test mode is enabled, consider @clerk/testing/playwright
})
```

#### Generic / Unknown Provider
```typescript
import { test as setup } from '@playwright/test'

const USER_AUTH_FILE = 'e2e/.auth/user.json'

setup('authenticate as user', async ({ page }) => {
  const email = process.env.E2E_USER_EMAIL
  const password = process.env.E2E_USER_PASSWORD

  if (!email || !password) {
    console.log('Skipping auth setup: E2E credentials not set')
    return
  }

  // Customize for your auth provider:
  await page.goto('/login')
  await page.getByLabel(/email/i).fill(email)
  await page.getByLabel(/password/i).fill(password)
  await page.getByRole('button', { name: /sign in|log in|submit/i }).click()
  await page.waitForURL('**/*', { timeout: 10000 })
  await page.context().storageState({ path: USER_AUTH_FILE })
})
```

---

## Phase 5: Validate

1. Verify the spec file was created:
   ```bash
   ls -la {output-file-path}
   ```

2. Run test discovery to confirm the file is valid:
   ```bash
   npx playwright test {output-file-path} --list 2>&1
   ```

3. If the feature exists and a dev server is running, optionally run the tests:
   ```bash
   npx playwright test {output-file-path} --project=chromium 2>&1 | tail -30
   ```

---

## Phase 6: Output

Report the results:

```
E2E Spec Generated

File: {output-file-path}
Feature: {feature name}
Routes: {/path1, /path2}
Auth: {required/not required}
Tests: {N} ({tier breakdown})
  Tier 1 (route loads): {N}
  Tier 2 (structural): {N}
  Tier 3 (behavioral): {N}
  Error states: {N}

Validation: {passed/failed}
{If failed: specific error and fix needed}

Next steps:
  1. Set E2E_USER_EMAIL and E2E_USER_PASSWORD in .env (if auth required)
  2. Run: npx playwright test --project=setup (to authenticate)
  3. Run: npx playwright test {file} (to run these tests)
```

View source on GitHub

Tags

README

Test Vision Module

Architecture

Usage

Full Test Suite Generation

Single Feature Spec

What Gets Generated

Test Infrastructure

Spec Files

CI/CD

Test-Driven Feature Development

Supported Auth Providers

Dependencies

Manual Installation

Will install

Dependencies

Required by

Included in presets

Install this module

Agent prompt

Native plugin marketplace

Manual, per file

Files

commands/test-vision.md

commands/e2e.md