[FIX]: Stop rendering "Pass" for regression tests with no test_result row

[x15sr71]

In raising this pull request, I confirm the following (please check boxes):

• I have read and understood the contributors guide.
• I have checked that another pull request for this purpose does not exist.
• I have considered, and confirmed that this submission will be valuable to others.
• I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
• I give this submission freely, and claim no ownership to its content.

My familiarity with the project is as follows (check one):

• I have never used the project.
• I have used the project briefly.
• I have used the project extensively, but have not contributed previously.
• I am an active contributor to the project.

---

Problem

Regression test rows on the test detail page render as "Pass" with empty Runtime/Exit Code columns when no test_result row was written for that RT. This contradicts both the category-level "Fail" header and the GitHub bot comments.

Production reproduction

• Test 9322 (Linux): https://sampleplatform.ccextractor.org/test/9322
• Test 9323 (Windows): https://sampleplatform.ccextractor.org/test/9323
• PR these ran for: ci: trigger sample platform test runs ccextractor#2279

For both tests, the DB has exactly 1 test_result row (RT 241, Hardsubx):

SELECT t.id, t.platform,
  (SELECT COUNT(*) FROM test_result WHERE test_id = t.id) AS tr_rows,
  (SELECT COUNT(*) FROM test_result_file WHERE test_id = t.id) AS trf_rows
FROM test t WHERE t.id IN (9322, 9323);
-- 9322 | linux   | 1 | 1
-- 9323 | windows | 1 | 1

But the UI shows all ~250 regression test rows for each, almost all labeled "Pass" with empty Runtime/Exit Code columns. The bot comments on PR #2279 report 0/86 Options, 0/27 General, 0/13 WTV, etc. — clearly contradicting the UI.

End-to-end chain

1. VM hit HTTP 504 timeouts mid-run on 9322/9323 (visible in the log file linked from the test page)
2. Only 1 RT (Hardsubx 241) finished and got a test_result row written
3. The other ~249 RTs in the configured set have no test_result row
4. get_test_results() in mod_test/controllers.py falls into the else branch for these RTs:

--python
else:
    outputs = RegressionTestOutput.query.filter(...).all()
    got = None
    if len(outputs) > 0:
        test_error = True
        got = 'error'
    category_test['files'] = [TestResultFile(-1, -1, -1, '', got)]

This correctly flags the category-level error (header shows "Fail") and injects a dummy TestResultFile with got='error'. But test.result remains None for these RTs (no row exists in test.results).

5. The template at templates/test/by_id.html:148 evaluates:

{% elif file.got is none or no_error.found or test.result.exit_code != 0 -%}
    Pass
{% elif file.got == "error" %}
    No output generated but there should be

In Jinja's default Undefined mode, None.exit_code becomes Undefined, and Undefined != 0 evaluates to True. The Pass elif fires before the got == "error" elif (which sits below it) can run.

Verified:

$ python3 -c "
from jinja2 import Environment, Undefined
e = Environment(undefined=Undefined)
print(repr(e.compile_expression('a.exit_code != 0')(a=None)))
print(repr(e.compile_expression('a.exit_code != 0')(a=type('x',(),{'exit_code':0})())))
"
True            # test.result is None → bug fires
False           # test.result has exit_code=0 → no bug

Why this is longstanding but only now noticed

The bug fires only when test.result is None — i.e., when a regression test has the dummy file-row injected by the controller's else branch but no real test_result row.

Most production tests have test_result rows for every RT, so the bug doesn't manifest. It becomes visible in edge cases:

• Test 9322/9323: only 1 of ~250 RTs got a test_result row (504 storms killed the VM mid-run) → ~249 rows render as false "Pass"
• BUG 2 (ServerComparer empty-path swallow, see ccx_testsuite#14): historically created 49,086+ missing comparison rows — every one rendered as false "Pass" on the test detail page
• BUG 4 (CEA-708 DISABLE_RUST since Mar 17, see #1122): all CEA-708 rows render as false "Pass" until [FIX]: point Artifact_names to (with migrations) builds to restore CEA-708 output #1122 fixes the underlying cause

The template has not been modified in this section for years, so the bug has likely been silent the whole time.

Fix

Two changes in templates/test/by_id.html:

1. Guard test.result.exit_code accesses with (test.result and ...) so the comparison is only evaluated when a real result row exists.
2. Reorder the elif chain so got == "error" is checked before the Pass elif. Defense-in-depth: even without the guards, the bug stays fixed because the correct branch runs first.

{% if test.result and test.result.exit_code != test.result.expected_rc %}
    ...Fail...
{% elif file.got == "error" %}
    No output generated but there should be
{% elif file.got is none or no_error.found or (test.result and test.result.exit_code != 0) -%}
    Pass
{% else %}
    ...Fail with diff link...
{%- endif %}

How to verify after deploy

Open https://sampleplatform.ccextractor.org/test/9322 after the deploy lands. Currently ~249 rows show "Pass" with empty Runtime/Exit Code. After the fix, those rows should render as "No output generated but there should be" (the controller already correctly flags them). Only RT 241 (Hardsubx, the one RT that actually ran) should remain as "Pass" with real runtime data.

Dependency on deploy unblock

Until the PEP 668 deploy block (fix-deploy-pep668 PR) lands, no new sample-platform code reaches the running process. This PR's effect won't be visible until that lands first.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud