🧪 Test Hub
testing: Demo target v3.0.0 · built-in demo site

DEMO-015 Freight: import a manifest (upload + download)

Edit Load test Duplicate

Builds a 30-row CSV with three deliberate mistakes, uploads it, waits for the server to check and price every row, verifies the three problems were caught, and downloads the processed file.

demodownloadfreightjobsupload version: 1.0 DEMO-015__freight_manifest_import.py group: Freight regression

Runs
7
Pass rate
100%
7 passed / 0 failed
Avg duration
6s
p95 6s
Estimate
6s
avg of last 10 judged runs
Flakiness
0.00
0 stable · 1 alternates
Current streak
7
passed

Last 7 results newest on the right — click a square to open that run

Duration per run point color = outcome; dashed = 7-run average; click a point to open the run

Step timing trends the same step compared across the last 7 runs — slowest steps first, one line each

How long it takes distribution of 7 runs — two humps mean two different behaviours hiding behind one average

fastest 5s · slowest 7s

Why this test failed grouped by message, last 90 days

No failures recorded. 👍

Outcomes by target version

What this test does plain language, derived from the code

  1. site = Freight(page, ctx).sign_in()
  2. manifest = ctx.artifacts_dir / "manifest.csv"
  3. manifest.write_text(build_manifest())
  4. page.goto(site.url("ops/import/"))
  5. page.set_input_files("#manifest-file", str(manifest))
  6. ⏱ Timed phase: process the manifest
  7. Click #upload-btn
  8. state = site.wait_for_job(timeout_s=90)
  9. assert state == "done", f"the import ended '{state}'"
  10. ok, bad = int(page.inner_text("#import-ok")), int(page.inner_text("#import-errors"))
  11. ctx.log(f"{ok} rows priced, {bad} with problems")
  12. Take a screenshot (“import-result”)
  13. assert (ok, bad) == (27, 3), f"expected 27 priced and 3 problems, got {ok} and {bad}"
  14. problems = " | ".join(page.locator("tr.row-error").all_inner_texts())
  15. for hint in ("did you mean", "is not a number", "cannot fly Express"):
  16. assert hint in problems, f"the import should have reported '{hint}': {problems}"
  17. results = site.download("#download-result", ctx.artifacts_dir / "manifest-results.csv")
  18. with open(results, newline="") as fh:
  19. statuses = [row["status"] for row in csv.DictReader(fh)]
  20. assert statuses.count("error") == 3 and statuses.count("ok") == 27, statuses
Show the code
"""Upload a file, let the server chew on it, download what comes back.

Builds a 30-row shipping manifest (CSV) with three deliberate mistakes,
uploads it through the page's file picker, waits for the server to check and
price every row, then asserts the three problems were caught -- each named
on its own row -- and downloads the processed file to compare.

File upload (set_input_files) and download (expect_download) are the two
things record-and-replay tools most often get wrong; here they are a few
readable lines.
"""
import csv
import io

from _lib.freight import Freight

GOOD = [
    ("Oakland, US", "Fresno, US"), ("Boston, US", "Philadelphia, US"),
    ("Dallas, US", "Houston, US"), ("San Diego, US", "Sacramento, US"),
    ("Austin, US", "El Paso, US"), ("Providence, US", "Albany, US"),
    ("San Jose, US", "Bakersfield, US"), ("Baltimore, US", "Hartford, US"),
    ("Fort Worth, US", "San Antonio, US"),
]
MISTAKES = {
    7: ("destination", "Sacremento"),       # typo -> "did you mean"
    15: ("weight_kg", "heavy"),             # not a number
    23: ("hazardous", "yes"),               # ...on Express: not allowed
}


def build_manifest():
    buf = io.StringIO()
    writer = csv.writer(buf, lineterminator="\n")
    writer.writerow(["reference", "origin", "destination", "weight_kg",
                     "pieces", "service", "hazardous"])
    for n in range(1, 31):
        origin, dest = GOOD[n % len(GOOD)]
        row = {"reference": f"QA-{7000 + n}", "origin": origin, "destination": dest,
               "weight_kg": str(20 + n * 3.5), "pieces": str(1 + n % 4),
               "service": ("standard", "economy", "express")[n % 3], "hazardous": "no"}
        if n in MISTAKES:
            column, value = MISTAKES[n]
            row[column] = value
            if column == "hazardous":
                row["service"] = "express"
        writer.writerow([row[c] for c in ("reference", "origin", "destination",
                                          "weight_kg", "pieces", "service", "hazardous")])
    return buf.getvalue()


def run(page, ctx):
    site = Freight(page, ctx).sign_in()
    manifest = ctx.artifacts_dir / "manifest.csv"
    manifest.write_text(build_manifest())

    page.goto(site.url("ops/import/"))
    page.set_input_files("#manifest-file", str(manifest))
    with ctx.timed("process the manifest"):
        page.click("#upload-btn")
        state = site.wait_for_job(timeout_s=90)
    assert state == "done", f"the import ended '{state}'"

    ok, bad = int(page.inner_text("#import-ok")), int(page.inner_text("#import-errors"))
    ctx.log(f"{ok} rows priced, {bad} with problems")
    ctx.screenshot("import-result")
    assert (ok, bad) == (27, 3), f"expected 27 priced and 3 problems, got {ok} and {bad}"
    problems = " | ".join(page.locator("tr.row-error").all_inner_texts())
    for hint in ("did you mean", "is not a number", "cannot fly Express"):
        assert hint in problems, f"the import should have reported '{hint}': {problems}"

    results = site.download("#download-result", ctx.artifacts_dir / "manifest-results.csv")
    with open(results, newline="") as fh:
        statuses = [row["status"] for row in csv.DictReader(fh)]
    assert statuses.count("error") == 3 and statuses.count("ok") == 27, statuses

All runs

RunStatusQueuedDuration VersionTriggerBatch
#490 passed 2026-09-25 07:00:10 5s 3.0.0 schedule schedule: Freight regression …
#370 passed 2026-09-24 22:05:21 6s 3.0.0 group group: Freight regression
#312 passed 2026-09-24 18:59:58 6s 3.0.0 cli-adopted 8 tests (terminal)
#304 passed 2026-09-24 18:57:59 5s 2.1.0 cli-adopted 8 tests (terminal)
#292 passed 2026-09-24 18:55:40 5s 2.0.0 cli-adopted 8 tests (terminal)
#153 passed 2026-09-24 18:52:00 5s 1.1.0 cli-adopted 8 tests (terminal)
#15 passed 2026-09-24 18:48:19 7s 1.0.0 cli-adopted 8 tests (terminal)