🧪 Test Hub
testing: Demo target v3.0.0 · built-in demo site

DEMO-014 Freight: optimize delivery routes (slow job)

Edit Load test Duplicate

Starts the route optimizer (a real server-side computation with live progress, 20-50 seconds depending on the release), waits for it, checks the plan and downloads it as CSV. The wait is timed, so the hub charts the optimizer's speed across releases.

demodownloadfreightjobsslow version: 1.0 DEMO-014__freight_route_optimizer.py group: Freight regression

Runs
7
Pass rate
100%
7 passed / 0 failed
Avg duration
28s
p95 46s
Estimate
28s
avg of last 10 judged runs
Flakiness
0.00
0 stable · 1 alternates
Current streak
7
passed

Last 7 results newest on the right — click a square to open that run

Duration per run point color = outcome; dashed = 7-run average; click a point to open the run

Step timing trends the same step compared across the last 7 runs — slowest steps first, one line each

How long it takes distribution of 7 runs — two humps mean two different behaviours hiding behind one average

fastest 19s · slowest 54s

Why this test failed grouped by message, last 90 days

No failures recorded. 👍

Outcomes by target version

What this test does plain language, derived from the code

  1. site = Freight(page, ctx).sign_in()
  2. page.goto(site.url("ops/optimizer/"))
  3. Choose “california” in #region
  4. page.fill("#stops", str(STOPS))
  5. page.fill("#trucks", str(TRUCKS))
  6. Click #start-optimizer
  7. Wait until #job-progress appears
  8. Take a screenshot (“optimizer-running”)
  9. ⏱ Timed phase: route optimization
  10. state = site.wait_for_job(timeout_s=170)
  11. assert state == "done", f"the optimization ended '{state}'"
  12. saving = float(page.get_attribute("#plan-saving", "data-pct"))
  13. ctx.log(f"optimized plan: {page.inner_text('#plan-best')} ({saving}% shorter than naive)")
  14. assert saving >= 25, f"the plan should be far shorter than the naive one; it saved only {saving}%"
  15. used = int(page.inner_text("#plan-trucks").split()[0])
  16. drawn = page.locator("svg polyline.route").count()
  17. assert drawn == used, f"{used} trucks are used but the map draws {drawn} routes"
  18. Take a screenshot (“route-plan”)
  19. plan = site.download("#download-result", ctx.artifacts_dir / "route-plan.csv")
  20. with open(plan, newline="") as fh:
  21. rows = list(csv.DictReader(fh))
  22. assert len(rows) == STOPS, f"the CSV plan should list {STOPS} stops, it lists {len(rows)}"
Show the code
"""Start a slow server-side job, wait for it, check the answer, keep the file.

The route optimizer plans delivery routes for a fleet of trucks: a real
computation that takes half a minute or so, with live progress on the page.
The test starts it, waits for it in ONE wait (the page polls the server and
reloads itself when the job ends), then checks the result makes sense and
downloads the plan as CSV into this run's artifacts.

The wait is wrapped in ctx.timed("route optimization"), so the hub charts how
long the optimizer takes, run after run and release after release -- which
is exactly how a release that quietly makes it twice as slow gets noticed.
"""
import csv

from _lib.freight import Freight

STOPS, TRUCKS = 25, 4


def run(page, ctx):
    site = Freight(page, ctx).sign_in()
    page.goto(site.url("ops/optimizer/"))
    page.select_option("#region", "california")
    page.fill("#stops", str(STOPS))
    page.fill("#trucks", str(TRUCKS))
    page.click("#start-optimizer")
    page.wait_for_selector("#job-progress")
    ctx.screenshot("optimizer-running")

    with ctx.timed("route optimization"):
        state = site.wait_for_job(timeout_s=170)
    assert state == "done", f"the optimization ended '{state}'"

    saving = float(page.get_attribute("#plan-saving", "data-pct"))
    ctx.log(f"optimized plan: {page.inner_text('#plan-best')} ({saving}% shorter than naive)")
    assert saving >= 25, f"the plan should be far shorter than the naive one; it saved only {saving}%"
    used = int(page.inner_text("#plan-trucks").split()[0])
    drawn = page.locator("svg polyline.route").count()
    assert drawn == used, f"{used} trucks are used but the map draws {drawn} routes"
    ctx.screenshot("route-plan")

    plan = site.download("#download-result", ctx.artifacts_dir / "route-plan.csv")
    with open(plan, newline="") as fh:
        rows = list(csv.DictReader(fh))
    assert len(rows) == STOPS, f"the CSV plan should list {STOPS} stops, it lists {len(rows)}"

All runs

RunStatusQueuedDuration VersionTriggerBatch
#489 passed 2026-09-25 07:00:10 19s 3.0.0 schedule schedule: Freight regression …
#369 passed 2026-09-24 22:05:21 21s 3.0.0 group group: Freight regression
#311 passed 2026-09-24 18:59:58 20s 3.0.0 cli-adopted 8 tests (terminal)
#303 passed 2026-09-24 18:57:59 27s 2.1.0 cli-adopted 8 tests (terminal)
#291 passed 2026-09-24 18:55:40 54s 2.0.0 cli-adopted 8 tests (terminal)
#152 passed 2026-09-24 18:52:00 27s 1.1.0 cli-adopted 8 tests (terminal)
#14 passed 2026-09-24 18:48:19 27s 1.0.0 cli-adopted 8 tests (terminal)