queued 2026-09-24 18:55:40 · started 18:55:42 · finished 18:55:45 · duration 2s · target v2.0.0 · chromium · trigger: cli-adopted · batch: 8 tests (terminal)
What went wrong
AssertionError: the home page looks different: similarity 0.6117 (min 0.97), 46.4% of pixels changed (max 2%). 02-what-changed.png in this run's screenshots marks it in red. If the change is intended, delete DEMO-017__baseline.png to re-bless the new look.
Technical details (full error)
AssertionError: the home page looks different: similarity 0.6117 (min 0.97), 46.4% of pixels changed (max 2%). 02-what-changed.png in this run's screenshots marks it in red. If the change is intended, delete DEMO-017__baseline.png to re-bless the new look.
Traceback (most recent call last):
File "/opt/pw-testhub/app/core/harness/run_test.py", line 737, in main
module.run(TimedPage(page, ctx), ctx)
File "/opt/pw-testhub/data/tests/DEMO-017__freight_visual_check.py", line 51, in run
assert score >= SSIM_MIN and changed <= DIFF_MAX, (
AssertionError: the home page looks different: similarity 0.6117 (min 0.97), 46.4% of pixels changed (max 2%). 02-what-changed.png in this run's screenshots marks it in red. If the change is intended, delete DEMO-017__baseline.png to re-bless the new look.
the page at the moment it failed
Activity replay 2 frames, captured only while the page was actually changing — idle waits are skipped automatically
+0.00s
Step timings follows the replay — click a step to see it there
| at | step | took |
|---|---|---|
| +0.75s | goto http://127.0.0.1:8000/demo/freight/ | 271 ms |
| +1.03s | wait_for_load_state networkidle | 444 ms |
| +1.47s | screenshot | 163 ms |
Times are when each step began, on the replay's clock. Add your own phases with
with ctx.timed("label"):.
Screenshots
Artifacts
| Run log | run.log |
| Playwright trace | trace.zip
open with: playwright show-trace trace.zip |
| Result JSON | result.json |
| Folder | results/DEMO-017/20260924-185541-r294/ |
Full log
Read from the S3 bucket (the last 20 KB; the whole file is the Run log link above).
[harness] test=DEMO-017 browser=chromium headed=False timeout=120s [harness] target: http://127.0.0.1:8000/demo/ [test] [ 0.75s] starting DEMO-017 [timing] goto http://127.0.0.1:8000/demo/freight/: 271ms [timing] wait_for_load_state networkidle: 444ms [timing] screenshot: 163ms [test] [ 2.18s] similarity 0.6117; 46.38% of pixels differ [test] [ 2.22s] FAILED: the home page looks different: similarity 0.6117 (min 0.97), 46.4% of pixels changed (max 2%). 02-what-changed.png in this run's screenshots marks it in red. If the change is intended, delete DEMO-017__baseline.png to re-bless the new look. [harness] security: 0 high, 1 medium, 1 low [harness] saved failure.png [harness] saved trace.zip [harness] result: failed
Security observations what the browser noticed while this ran — not a penetration test
| Severity | Finding | What to do |
|---|---|---|
| medium | The application is served over plain HTTP http://127.0.0.1:8000/demo/ is not HTTPS, so anything typed into it travels the network in the clear and can be modified in transit. |
On an isolated lab network this may be a deliberate choice -- if so, note it and move on. Anywhere else, terminate TLS in front of the application. |
| low | Response advertises software version (server) server: pw-testhub -- this tells an attacker exactly what to look up known vulnerabilities for. |
Suppress or blank the server header at the proxy. |
| info | Cookie 'csrftoken' is readable by JavaScript (expected) This looks like a CSRF token, which the page's own JavaScript has to read in order to send it back. HttpOnly would break that. Nothing to do -- listed so you know it was checked and not missed. |
No action. Do check that SESSION cookies are HttpOnly. |
All of these appear together on the Security page.