Skip to content

fix(gfql/cypher): neo4j round() tie-breaking + lower/upper GQL aliases (#1673)#1677

Merged
lmeyerov merged 9 commits into
masterfrom
dev/gfql-cypher-std-conformance
Jul 5, 2026
Merged

fix(gfql/cypher): neo4j round() tie-breaking + lower/upper GQL aliases (#1673)#1677
lmeyerov merged 9 commits into
masterfrom
dev/gfql-cypher-std-conformance

Conversation

@lmeyerov

@lmeyerov lmeyerov commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Stacked on #1675. Result of a deep standards re-vet of the #1673 dialect extensions against primary sources — the current neo4j Cypher manual, neo4j's shipped Cypher 5/25 parser source, the openCypher TCK, and the ISO-derived GQL grammar + neo4j's GQL-conformance appendix. Full findings table posted on #1673.

Fixes

1. round() tie-breaking — was a wrong answer vs neo4j. We shipped numpy/polars half-to-even (round(2.5) → 2.0). Neo4j's documented semantics: precision 0 (and the 1-arg form) rounds ties toward +∞ (round(-1.5)-1.0, manual Ex. 8/10); precision > 0 rounds ties away from zero (HALF_UP: round(-1.55, 1)-1.6, Ex. 11). Fixed on pandas/cuDF (floor(x+0.5) / scaled HALF_UP) and polars (floor(x+0.5) for p=0; native round(mode="half_away_from_zero") for p>0 — a manual scale/divide formula picks up 1-ulp noise from polars' reassociating optimizer). Parity-tested on both engines against the manual's example values. (Note: the neo4j args-table "(default: UP)" contradicts its own Example 10 — docs bug; the examples are authoritative.)

2. lower() / upper() aliases added — ISO GQL mandatory §20.24 character-string functions; neo4j accepts both spellings as GQL-conformance aliases of toLower/toUpper. A GQL-aligned dialect should too.

Spec records (no code change, see #1673 addendum)

  • ^ settled as LEFT-associative for the eventual re-add: the openCypher TCK pins left (4^(3*2)^3 = (4^6)^3) and neo4j's shipped parser left-folds Pow (Cypher25Parser.g4 + astBinaryFold=foldLeft) — the manual's "Right to left" precedence row is a neo4j docs bug.
  • XOR correction: ISO GQL does have XOR (optional GE07; precedence groups with OR).
  • GRAPH g = GRAPH { … } and CREATE GFQL INDEX confirmed as documented extensions (GQL §10.1 takes graph references only; GQL has zero index DDL — neo4j's own CREATE INDEX is likewise beyond-GQL).
  • LIKE/BETWEEN absence from GQL re-verified hard (grammar's only LIKE is CREATE GRAPH TYPE … LIKE g).

Tests

Neo4j-manual example values asserted on pandas and polars (test_round_neo4j_tie_breaking, test_lower_upper_gql_aliases, updated scalar cases); 1618 tests pass across expr-parser/lowering/row-pipeline/polars suites; ruff + mypy clean. cuDF lane: the fix uses the same engine-portable pattern as floor/ceil (cuDF-native .floor() else np.floor). GPU parity verified on dgx (GB10, --gpus all, 2026-07-05): full polars lane + conformance matrix + GPU suite = 1807 passed, 0 skipped/failed — the hardened round kernel + lower/upper aliases parity-green on cuDF and polars-gpu vs the pandas oracle.

Review hardening (4-wave converged adversarial review, 2026-07-05)

A multi-wave findings-mode review (artifacts: plans/viz-filter-pipeline/waves/pr1677/) found and fixed, dgx-verified: the polars extra's floor corrected to >=1.29 (Expr.round(mode=) shipped in py-1.29.0, not 1.5 — older installs crashed with a raw TypeError); round(x, p>308) is now the identity on BOTH engines (10.0**p overflow guarded on pandas; polars' quantizing [309,325] window + u32 crash above 2^32 guarded on polars); polars round(p>0) normalizes -0.0 like the pandas kernel (pinned by a sign-bit test — value equality can't see it); deterministic tie/hazard matrix cases so exact ties (±1.25 at p=1, 2^51+0.5, the JDK 1-ulp-below-tie case, 1e308, p=400) actually reach cuDF/polars-gpu; the neo4j BigDecimal decimal-string deviation documented (round(2.675, 2) = 2.67 binary-double here vs 2.68 in neo4j); warning hygiene on the guarded overflow paths. Reviewers verified the p=0 kernel ulp-correct and byte-identical cross-engine, and the tie semantics against both the neo4j manual and CypherFunctions.java. Waves 3–4 converged at zero BLOCKER/IMPORTANT.

🤖 Generated with Claude Code

@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from b6bb4e6 to 197ff1e Compare July 2, 2026 16:49
lmeyerov added a commit that referenced this pull request Jul 2, 2026
…in _cudf_regex_prep

The cuDF =~ flags commit annotated _cudf_regex_prep with "tuple[object, bool]",
which mypy rejects on py3.8/3.9 (builtin generics need 3.9+/3.10+); broke the
python-lint-types (3.8, 3.9) lanes on #1675/#1677. Use typing.Tuple.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from 197ff1e to 3ca652e Compare July 2, 2026 16:54
lmeyerov added a commit that referenced this pull request Jul 2, 2026
…lane

The polars-parametrized cypher tests (numeric fns, toLower, =~ parity in
graphistry/tests/compute/gfql/cypher/test_lowering.py) never executed in CI:
the core lane has no polars installed (importorskip -> skip) and the polars
lane's fixed file list excluded the file. So the polars numeric lowerings this
branch adds were CI-untested and their changed lines uncovered (coverage gate
76.9% < 80). Add the file (-k polars) to bin/test-polars.sh and the ci.yml
polars coverage step (--cov-append). Moved from the stacked #1677 to here — it
belongs with the lowerings it tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from 3ca652e to 21472b4 Compare July 2, 2026 17:24
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-viz-ops branch from f997217 to 31054a8 Compare July 2, 2026 23:40
lmeyerov added a commit that referenced this pull request Jul 2, 2026
…in _cudf_regex_prep

The cuDF =~ flags commit annotated _cudf_regex_prep with "tuple[object, bool]",
which mypy rejects on py3.8/3.9 (builtin generics need 3.9+/3.10+); broke the
python-lint-types (3.8, 3.9) lanes on #1675/#1677. Use typing.Tuple.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lmeyerov added a commit that referenced this pull request Jul 2, 2026
…lane

The polars-parametrized cypher tests (numeric fns, toLower, =~ parity in
graphistry/tests/compute/gfql/cypher/test_lowering.py) never executed in CI:
the core lane has no polars installed (importorskip -> skip) and the polars
lane's fixed file list excluded the file. So the polars numeric lowerings this
branch adds were CI-untested and their changed lines uncovered (coverage gate
76.9% < 80). Add the file (-k polars) to bin/test-polars.sh and the ci.yml
polars coverage step (--cov-append). Moved from the stacked #1677 to here — it
belongs with the lowerings it tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from 21472b4 to ccede7e Compare July 2, 2026 23:40
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-viz-ops branch from 31054a8 to 8843b0f Compare July 4, 2026 20:24
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…in _cudf_regex_prep

The cuDF =~ flags commit annotated _cudf_regex_prep with "tuple[object, bool]",
which mypy rejects on py3.8/3.9 (builtin generics need 3.9+/3.10+); broke the
python-lint-types (3.8, 3.9) lanes on #1675/#1677. Use typing.Tuple.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…lane

The polars-parametrized cypher tests (numeric fns, toLower, =~ parity in
graphistry/tests/compute/gfql/cypher/test_lowering.py) never executed in CI:
the core lane has no polars installed (importorskip -> skip) and the polars
lane's fixed file list excluded the file. So the polars numeric lowerings this
branch adds were CI-untested and their changed lines uncovered (coverage gate
76.9% < 80). Add the file (-k polars) to bin/test-polars.sh and the ci.yml
polars coverage step (--cov-append). Moved from the stacked #1677 to here — it
belongs with the lowerings it tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch 2 times, most recently from c7c889f to 401432b Compare July 4, 2026 20:55
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-viz-ops branch from 6d65aad to 17b127c Compare July 4, 2026 22:24
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…in _cudf_regex_prep

The cuDF =~ flags commit annotated _cudf_regex_prep with "tuple[object, bool]",
which mypy rejects on py3.8/3.9 (builtin generics need 3.9+/3.10+); broke the
python-lint-types (3.8, 3.9) lanes on #1675/#1677. Use typing.Tuple.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…lane

The polars-parametrized cypher tests (numeric fns, toLower, =~ parity in
graphistry/tests/compute/gfql/cypher/test_lowering.py) never executed in CI:
the core lane has no polars installed (importorskip -> skip) and the polars
lane's fixed file list excluded the file. So the polars numeric lowerings this
branch adds were CI-untested and their changed lines uncovered (coverage gate
76.9% < 80). Add the file (-k polars) to bin/test-polars.sh and the ci.yml
polars coverage step (--cov-append). Moved from the stacked #1677 to here — it
belongs with the lowerings it tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from 401432b to 24b50ab Compare July 4, 2026 22:24
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…in _cudf_regex_prep

The cuDF =~ flags commit annotated _cudf_regex_prep with "tuple[object, bool]",
which mypy rejects on py3.8/3.9 (builtin generics need 3.9+/3.10+); broke the
python-lint-types (3.8, 3.9) lanes on #1675/#1677. Use typing.Tuple.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-viz-ops branch from 17b127c to e9ba70f Compare July 4, 2026 22:57
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…lane

The polars-parametrized cypher tests (numeric fns, toLower, =~ parity in
graphistry/tests/compute/gfql/cypher/test_lowering.py) never executed in CI:
the core lane has no polars installed (importorskip -> skip) and the polars
lane's fixed file list excluded the file. So the polars numeric lowerings this
branch adds were CI-untested and their changed lines uncovered (coverage gate
76.9% < 80). Add the file (-k polars) to bin/test-polars.sh and the ci.yml
polars coverage step (--cov-append). Moved from the stacked #1677 to here — it
belongs with the lowerings it tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from 24b50ab to 73563ee Compare July 4, 2026 22:57
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…in _cudf_regex_prep

The cuDF =~ flags commit annotated _cudf_regex_prep with "tuple[object, bool]",
which mypy rejects on py3.8/3.9 (builtin generics need 3.9+/3.10+); broke the
python-lint-types (3.8, 3.9) lanes on #1675/#1677. Use typing.Tuple.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-viz-ops branch from e9ba70f to 4384336 Compare July 4, 2026 23:39
lmeyerov added a commit that referenced this pull request Jul 4, 2026
…lane

The polars-parametrized cypher tests (numeric fns, toLower, =~ parity in
graphistry/tests/compute/gfql/cypher/test_lowering.py) never executed in CI:
the core lane has no polars installed (importorskip -> skip) and the polars
lane's fixed file list excluded the file. So the polars numeric lowerings this
branch adds were CI-untested and their changed lines uncovered (coverage gate
76.9% < 80). Add the file (-k polars) to bin/test-polars.sh and the ci.yml
polars coverage step (--cov-append). Moved from the stacked #1677 to here — it
belongs with the lowerings it tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch 2 times, most recently from 7695d77 to 4c5becc Compare July 5, 2026 01:07
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from 4c5becc to fbc9f27 Compare July 5, 2026 01:42
lmeyerov and others added 8 commits July 5, 2026 14:50
#1673 standards vetting)

Standards re-vet vs neo4j Cypher 25 (manual + shipped parser source) and ISO GQL
(opengql/grammar + neo4j GQL-conformance appendix) found one wrong-answer and one
missing conformance alias in the shipped numeric/string functions:

1. round() ties DIVERGED: we used numpy/polars half-to-even (round(2.5) -> 2.0),
   but neo4j's documented semantics are precision 0 (and 1-arg) = ties toward
   +inf (round(-1.5) = -1.0, manual Ex. 8/10) and precision > 0 = ties away from
   zero (HALF_UP: round(-1.55, 1) = -1.6, Ex. 11). Fixed on pandas/cuDF
   (floor(x+0.5) / scaled HALF_UP) and polars (floor(x+0.5) for p=0; native
   round(mode='half_away_from_zero') for p>0 — a manual scale/divide formula
   picks up 1-ulp noise from polars' reassociating optimizer). Parity-tested
   against the manual's example values on both engines.

2. lower()/upper() added as aliases of toLower()/toUpper() — ISO GQL mandatory
   character-string functions (§20.24); neo4j accepts both spellings as its
   GQL-conformance aliases.

Also records the settled `^` spec (LEFT-associative: openCypher TCK + neo4j's
shipped parser both left-fold; the manual's "right to left" row is a docs bug)
for the deferred re-add, and corrects the XOR note (GQL HAS XOR, GE07).

1618 tests pass; ruff + mypy clean; CHANGELOG + docs updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tive-precision decline, CI polars lane

Adversarial review of the round() conformance fix (4 IMPORTANT findings):
- I2/S2: replace floor(x+0.5) with a floor+frac kernel on both engines — the
  +0.5 addition itself rounds up when x sits 1 ulp below a tie (JDK-6430675),
  e.g. round(0.49999999999999994) must be 0.0 and round(0.0499…96, 1) -> 0.0
  (pandas previously disagreed with polars/neo4j here). Also: scaled-overflow
  identity (round(1e300, 20) = 1e300, not inf), -0.0 normalized (+0.0).
- I1: negative precision now declines honestly on both engines (neo4j raises;
  polars previously crashed with a raw OverflowError, pandas silently computed).
- I3: the polars-parametrized cypher test_lowering cases never ran in CI (core
  lane has no polars; polars lane's file list excluded the file) — added
  cypher/test_lowering.py -k polars to bin/test-polars.sh + the ci.yml coverage
  step (--cov-append).
- I4: setup.py polars extra floor >= 1.5 (round(mode=) kwarg requirement).

+ulp/inf/negative-precision tests (engine-parametrized, importorskip-guarded).
1620 tests pass; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Same ledger-driven completion as the parent commit: the aliases entered
GFQL_SCALAR_FUNCTIONS here, so they get real matrix parity cases rather than
waivers. dgx: matrix + ledger 269 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… identity, -0.0 normalize, tie/hazard matrix

Correct extras floor (Expr.round(mode=) shipped py-1.29.0, not 1.5); round(x,p>308)
= identity on both engines (10.0**p overflow guarded; was err-vs-identity divergence);
polars p>0 +0.0 zero-sign normalize (sign-bit-pinned test); deterministic tie/hazard
matrix case (ties never reached GPU engines); BigDecimal deviation documented.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…p>0 ties in hazard set, warning hygiene

Polars mirrors the pandas p>308 identity (+0.0): its native identity starts at
p>=326 (quantizing [309,325] window) and p>=2^32 is a raw PyO3 OverflowError;
hazard set gains ±1.25 (exact p=1 ties — no prior value tied at p>0) + 2^51+0.5
(representable huge tie), drops the dead 5e15+0.5; sign-bit test runs all engines;
errstate(over=ignore) on the kernel's guarded scale multiply and the test sig's
round(6).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e keep-both

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…erflow window

over= alone still warned on the inf-inf subtract (invalid value encountered);
values unaffected (both guarded to identity).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ave-4 suggestion)

±inf input warns identically on the p=0 tie subtract; hoist one errstate over
both branches (values unchanged, all guarded).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-viz-ops branch from c61feb7 to 38be679 Compare July 5, 2026 22:10
lmeyerov added a commit that referenced this pull request Jul 5, 2026
…in _cudf_regex_prep

The cuDF =~ flags commit annotated _cudf_regex_prep with "tuple[object, bool]",
which mypy rejects on py3.8/3.9 (builtin generics need 3.9+/3.10+); broke the
python-lint-types (3.8, 3.9) lanes on #1675/#1677. Use typing.Tuple.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lmeyerov added a commit that referenced this pull request Jul 5, 2026
…lane

The polars-parametrized cypher tests (numeric fns, toLower, =~ parity in
graphistry/tests/compute/gfql/cypher/test_lowering.py) never executed in CI:
the core lane has no polars installed (importorskip -> skip) and the polars
lane's fixed file list excluded the file. So the polars numeric lowerings this
branch adds were CI-untested and their changed lines uncovered (coverage gate
76.9% < 80). Add the file (-k polars) to bin/test-polars.sh and the ci.yml
polars coverage step (--cov-append). Moved from the stacked #1677 to here — it
belongs with the lowerings it tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lmeyerov lmeyerov force-pushed the dev/gfql-cypher-std-conformance branch from 3e2c0ea to ccd440e Compare July 5, 2026 22:11
… bare list)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Base automatically changed from dev/gfql-cypher-viz-ops to master July 5, 2026 23:10
@lmeyerov lmeyerov merged commit 96c30e4 into master Jul 5, 2026
75 checks passed
@lmeyerov lmeyerov deleted the dev/gfql-cypher-std-conformance branch July 5, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant