Acme Freight
v3.0.0

A guided tour: what the Test Hub can do

Acme Freight is a make-believe shipping company that exists to be tested. It runs entirely on this server — maps, data and all — so everything below works on an air-gapped network. Twenty minutes, start to finish.

  1. Meet the site

    Track AF-100001, get a quote (promo code SPRING10), then sign in to the staff portal as tester / demo-password and start the route optimizer: it plans delivery routes on the server and you can watch it improve.

  2. Run the regression suite

    In the Test Hub, open Groups → Freight regression → Run. Watch the live view from the dashboard. When a run finishes, open it: video, Playwright trace, the activity reel (it skips the boring waits), a screenshot per step, and the time every step took.

  3. Ship new releases, and let the hub notice

    Open the Release console, deploy the next release, run the group again. Repeat through 3.0.0. What each release breaks or improves:

    ReleaseWhat changedWhere the hub shows it
    1.0.0 LaunchNothing is broken -- this is the baselineRun the Freight regression group once: every test passes, and every later version is compared against these numbers
    1.1.0 Faster searchSearch, tracking and imports really are fasterAnalytics speed table and the per-step trend charts: durations drop for DEMO-010, DEMO-013 and DEMO-015
    2.0.0 The redesignEvery page looks differentDEMO-017 (visual comparison) fails until you re-bless its baseline -- delete DEMO-017__baseline.png in the tests folder
    The route optimizer quietly got about 2x slowerAnalytics speed table + DEMO-014's 'route optimization' step-timing trend
    Saving a note fails every third time (HTTP 503)DEMO-013 fails some runs but not others: its pass rate drops -- the flaky-test signal on Metrics and Analytics
    The public API renamed 'eta' to 'estimated_delivery' -- a breaking change nobody announcedDEMO-020 (API contract) fails: every client reading 'eta' would break
    Accessibility went backwards: the pages lose their language, the skip link is hidden from screen readers, the map loses its text alternative and the hint text is too faint to readDEMO-022 (accessibility scan) fails, naming each broken WCAG rule per page; its full report is saved with the run
    2.1.0 HotfixSign-in no longer rotates the session id (session fixation)DEMO-012's authenticated checks fail; the Security page lists it
    The Content-Security-Policy and X-Frame-Options headers were droppedSecurity page: missing-header findings on every page
    The 'usage analytics' script calls analytics.acme-metrics.invalidSecurity page: external-request, HIGH -- on an air gap that is both a broken page and a path out of the network
    Promo codes are applied twice (20% off instead of 10%)DEMO-011 fails, and its message names both amounts -- the API quotes (DEMO-020) are wrong the same way
    The public API's rate limiter was switched offDEMO-020 lists it with the other API problems: six calls in a row, no HTTP 429
    The faster reset emails skipped a step: a password-reset link keeps working after it has been usedDEMO-023 fails: any old reset email is still a key to the account
    2.0's accessibility problems are all still thereDEMO-022 still fails
    3.0.0 StableEvery planted problem is fixedThe whole Freight regression group is green again (after DEMO-017's baseline was re-blessed for the new design)
    The optimizer is ~30% faster than 1.0, search the fastest yetAnalytics speed table: 'faster' across the board
    The API sends BOTH 'eta' and 'estimated_delivery' (eta deprecated)DEMO-020 green again: old clients keep working, new ones migrate
    Accessibility and single-use reset links are fixedDEMO-022 and DEMO-023 green again

    Then open Analytics: every test × every release, pass rates, and which tests got slower or faster. Security shows what the 2.1.0 hotfix quietly broke, and Metrics has the per-test trends.

  4. Beyond clicking pages: an API, drag-and-drop, live data

    Run the Freight platform group. DEMO-020 tests the public REST API with no page at all — keys, the JSON contract, exact money, the rate limit — and lists every broken promise in one run (deploy 2.0.0 and it has one to report). DEMO-021 drags trucks onto the dock schedule and checks that the yard's rules refuse a hazardous load at the wrong door. DEMO-019 picks a truck on the live fleet map and waits for exactly that arrival.

  5. The checks people still do by hand

    Run the Freight extras group. DEMO-022 scans five pages for WCAG 2.1 AA accessibility problems (what Section 508 asks for) with the same engine commercial scanners use — it runs offline — and saves a full report with the run; the 2.0 redesign fails it while looking perfectly fine. DEMO-023 resets a password through email: it waits for the message in the Mailbox, follows the link, and checks the link then stops working (on 2.1.0 it does not). DEMO-024 opens a shipping label in a new tab and reads its barcode the way a depot scanner would.

  6. Break it on purpose

    In the Release console, Take the site down, then run Freight smoke: every test fails within seconds and says the site is failing — not that a button went missing. The status page stays up and says so too. Try Make it slow as well, then All clear.

  7. Put it under load

    Open DEMO-018 → Load → Quick smoke. Real browsers hammer the tracking page; the result names the step that degraded. Try it on 1.0.0 and on 1.1.0.

  8. Write your own test in two minutes

    Open the Recorder on this site: the hub's own browser opens Acme Freight, you click through a quote on the picture, right-click anything to add a check, and save it as a new test — in Python or TypeScript.

  9. Or drive it from a terminal

    testhub run DEMO-014 queues a run and returns at once; testhub watch follows it; testhub tui is the whole hub in a terminal window.