Running the Consultant Audit Workbench locally

The Consultant Audit Workbench is the internal GUI for creating and managing accessibility assessments. It is separate from the dashboard. Use the workbench to create assessments, upload artifacts, track progress, and hand off results.

What the workbench does

• Create a new web or native (Android/iOS) accessibility assessment
• Choose crawl settings (web) or upload artifact bundles (native)
• Select a saved auth profile for authenticated flows
• Start, cancel, or monitor an assessment
• Reassess from an existing configuration
• Compare assessments via dashboard diff
• Hand off completed results to the dashboard

Prerequisites

• The A11YSmith monorepo checked out locally
• pnpm installed
• Project dependencies installed
• Any environment variables required by your auth profiles available in your shell

Start the workbench

From the repo root:

pnpm install

pnpm --filter @a11ysmith/workbench dev

The workbench starts on port 4322 by default. Open the local URL shown in the terminal.

Architecture

The workbench comprises three packages:

Create a web audit

1. Open the workbench home page.
2. Select New Audit.
3. Select the Web platform (selected by default).
4. Enter a start URL.
5. Enable crawl only if you want multi-page discovery.
6. Adjust crawl settings as needed.
7. Select a saved auth profile if the target flow requires sign-in.
8. Leave sanitization enabled unless you have a specific internal reason to turn it off.
9. Click Create and Start Audit.

Create a native artifact audit

1. Select New Audit.
2. Select the Android or iOS platform.
3. Choose the audit mode (artifact only, source only, hybrid, or manual assist).
4. Enter target identifiers (screen names, activity names).
5. Click Create Draft and Upload Artifacts to create a draft run.
6. On the upload page, upload artifact files (view hierarchies, test output, lint output, source files).
7. Validate the artifact bundle.
8. Start the run once validation passes.

See Native Artifacts for artifact types, upload limits, and validation rules.

Monitor run status

The workbench supports the full run lifecycle: draft, queued, starting, running, completed, completed with warnings, failed, canceled, and partial success.

Use the assessment detail page to:

• Monitor progress
• Review status changes
• Read terminal errors
• Cancel an active assessment
• Open the generated report

Reassess and compare

From a completed assessment, click Reassess to create a new draft with the same configuration. Click Compare in Dashboard to open the diff view between the current assessment and its baseline.

See Reassess and Compare for details on lineage tracking, config preservation, and diff URL generation.

Auth profiles

The workbench supports saved auth profiles for storage state, scripted login, and headers-based authentication. Secret values are resolved from environment variables at runtime.

See Auth Profiles for strategy details, security model, and API reference.

Where local data is stored

Override the data directory with the A11YSMITH_DATA_DIR environment variable.

Troubleshooting

The run does not start

• Confirm dependencies are installed.
• Confirm the start URL is valid (web) or artifact bundle is validated (native).
• Confirm any selected auth profile still exists.

The assessment fails immediately

• Check the assessment detail page for the terminal error.
• Verify that any required environment variables for the selected auth profile are present.
• For native assessments, check for ARTIFACT_VALIDATION_FAILED or ARTIFACT_PARSE_FAILED error codes.

Findings do not appear in the dashboard

• Confirm the assessment reached a terminal success state.
• Confirm the report was generated and handed off to the dashboard.
• Confirm you opened the JSON-backed dashboard result, not just the assessment detail page.

Native artifact upload fails

• Individual files are limited to 50 MB, total bundle to 200 MB, maximum 50 files.
• Check that artifact types match the platform.
• Ensure artifact_only mode includes at least one view_hierarchy file.