feat(gfql): native lazy Polars engine — collect-once traversals + cypher row pipeline + followups (multi-hop, to_fixed_point, undirected, more predicates, NIE→native) by lmeyerov · Pull Request #1660 · graphistry/pygraphistry

lmeyerov · 2026-06-29T03:17:17Z

Note: Supersedes #1648, which GitHub auto-closed when its base branch dev/gfql-opt-base was deleted on the #1652 merge (the close was a mechanical side-effect, not a rejection — mergedAt=null). This is the same head branch dev/gfql-polars-engine, rebased onto master (the merged #1652/#1657/#1656 commits dropped). Reopen was not possible (base branch gone + head rebased).

Summary

Native CPU Polars execution engine for GFQL (Engine.POLARS, opt-in via engine='polars') — combines the former two-PR split (hop/chain traversals + cypher row pipeline) into one cohesive CPU-engine PR. The production pandas/cuDF paths are untouched; engine='auto' with Polars input still coerces to pandas as before.

Note: this PR now includes the GPU execution target (engine='polars-gpu', cudf_polars) — former stacked PR #1655 was folded in by fast-forward (GitHub shows it merged): the engine PR already contained the lazy collect-once machinery the GPU target completes ("one engine, two targets"), so the boundary cut a single feature in half. Also folds the former #1667 (native followups — see the Followups section). Base: #1666 (3VL null-semantics fix).

GPU target — `engine='polars-gpu'` (folded #1655)

The same lazy plan, executed on the RAPIDS cudf_polars backend via ONE collect_all per hop (collect-once is what makes GPU pay off: per-op eager GPU collect was a measured regression). Explicit opt-in only (AUTO never selects it). GPU-or-error (raise_on_fail=True): a GPU-incapable plan node raises (pointing at engine='polars') — never silent-CPU mislabeled as GPU. Includes opt-in CPU streaming collect (GFQL_POLARS_CPU_STREAMING=1) for large batch traversals. Validated by engine='polars-gpu' == engine='polars' differential parity across the conformance corpus + traversals (2.84× single-hop GPU win @1m vs CPU).

Followups — native completions (folded #1667)

The former stacked PR #1667 was folded in (GitHub shows it merged) — it completes the native surface rather than adding a separable feature:

Native variable-length min_hops>1 forward/reverse traversals (no pandas bridge) — the subtle min_hops NODE-output rule proven against the pandas oracle (500/500 chain fuzz on both polars and polars-gpu).
Native get_degrees / get_indegrees / get_outdegrees (parity vs ComputeMixin), incl. via the Cypher CALL surface.
Wider native cypher coverage: toFloat, collect/collect(DISTINCT), WHERE … IN, size/substring, to_fixed_point, more predicates — each parity-validated or honest-NIE.
count(*) short-circuit: a lone RETURN count(*) reads table height (or sums a boolean mask) in one reduction instead of materializing + group_by — engine-polymorphic across all four engines.

Traversals — `hop()` / `chain()`

Native vectorized BFS via semi/anti joins (no per-row Python). Forward/reverse/undirected single-hop, directed multi-hop chains, node/edge filter dicts and predicates (lowered to pl.Expr), edge_match/source_node_match/destination_node_match, target_wave_front, alias names. Deferred (honest NotImplementedError): variable-length/multi-hop edges, undirected edges in multi-edge chains, hop labels, node query=.

Cypher row pipeline — `MATCH … RETURN`

NO CHEATING: every query runs natively on Polars or raises an honest NotImplementedError pointing at engine='pandas' — never a silent pandas bridge. Native: frame ops (rows/limit/skip/distinct/drop_cols), select/with_/return_ projection (cypher-expr-AST → pl.Expr: property/arithmetic/comparison/boolean/literal + coalesce/abs), where_rows (OR/NOT WHERE, Kleene 3-valued), order_by, group_by (count/sum/avg/min/max), unwind (literal cross-join), property/expr result projection, int/string/bool entity-text (pl.concat_str). Honestly deferred → NIE: cross-entity same-path WHERE, multi-entity binding_ops, float/temporal/nested entity-text, exotic exprs.

Conformance hardening

Driven by the cross-repo Cypher TCK differential (pandas-vs-polars). Every fix either matches pandas natively or declines honestly (NO-CHEATING — no silent bridge, no wrong answer):

IEEE NaN comparison semantics — 0.0/0.0 > 1 etc.: Polars treats NaN as the largest value; now masked to the IEEE/pandas answer (& ~is_nan for ordering/=, | is_nan for <>), driven by the lowered expression's output dtype (robustly covers int/int→float division + function results).
numeric-vs-string comparison (n.val > 'a') — would Rust-panic; now NIE (recurses into AllOf/Between, covers Categorical/Enum + arithmetic on all-null→String columns).
ISO-temporal ordering / temporal arithmetic (time({...}) > time({...}), a.time + duration({...})) — were lexicographic/string-concat wrong answers; now NIE (narrowed to ordering of two temporal literals; =/<> stay native).
temporal-constructor-string property + heterogeneous (mixed-type) column projections — NIE (clear message naming the column) instead of leaking raw constructor text / cryptic ArrowInvalid.
UNION DISTINCT — engine-aware Engine.df_unique (Polars unique(maintain_order=True)) instead of the pandas-only drop_duplicates crash.
OPTIONAL MATCH — absent whole-entity renders null (not '()'); null-row-fill alignment shape declines honestly (NIE) instead of a misleading validation error.
3-valued boolean over null literals (null AND null, NOT null) — AND/OR/NOT cast to pl.Boolean so Kleene logic evaluates instead of raising.
label match on the reserved labels List column — list.contains (membership) instead of a List→String cast crash; user List properties decline honestly.
predicate pandas-bridge removed — an unlowerable filter_by_dict predicate now raises NIE (was silently .to_pandas()-bridging); native lowering widened (AllOf, IsNull/NotNull, case-insensitive STARTS/ENDS WITH).
WITH-scalar MATCH re-entry — engine-aware (pl.row/with_columns) instead of a pandas .iloc crash.

POLARS_ENGINES (POLARS + POLARS_GPU) is introduced here so the engine-aware helpers are self-contained at this layer.

Validation

Differential parity vs the pandas engine (hop + chain suites + seeded fuzzer + a TCK-style cypher conformance lane with NULL/3-valued-logic + a DEFERRED list asserting deferred queries raise rather than bridge). Full graphistry/tests/compute/gfql/ suite green (incl. the 1610-test cypher dir).

Cross-repo Cypher TCK, polars arm: the differential pandas-vs-polars lane is clean — 0 wrong-answers across the full TCK (every scenario either matches pandas or honestly declines; 387 honest declines). This is the headline correctness guarantee: the Polars engine never silently disagrees with pandas.

Perf (interleaved, 1M nodes, each engine on its native-frame graph, all native)

Polars wins 5.6–38× across the surface: RETURN n ~38×, ORDER BY ~17×, traversals 6–7.5×, projections/aggregations/DISTINCT 5.6–6.9×. Plus eager fast paths (node-only / single-hop / unconstrained 1-hop) that move the polars>pandas crossover below ~100K for real viz/crossfilter shapes.

Recent refinements (review-driven)

Post-fold polish — ergonomics, correctness, and typing:

cuDF→polars conversion is dtype-lossless: converts via Arrow (cuDF's native interchange) instead of cuDF→pandas→polars, which double-converted and was lossy (nullable Int64→float64+NaN). Verified on dgx; also removes a device→host→device round trip for polars-gpu.
validate/warn convention on engine conversion: Engine.df_to_engine(..., validate=, warn=) — strict (polars default) raises on an un-representable mixed-type column, autofix coerces+warns (matching cuDF + the plot()/upload() boundary). Both Engine.py from_pandas converters unified onto the protocol.
Polars execution config is Python-settable + live: set_cpu_streaming / set_gpu_executor (+ the public GPU_EXECUTORS options) — previously env-only (GFQL_POLARS_*) and frozen at import; documented in the engine-selection guide (docs PR).
Typing + dead-code sweep: removed the getattr(g, "_nodes", None) no-op anti-pattern on declared Plottable attrs (result_postprocess / reentry / row pipeline / gfql_unified / call procedures); typed the polars-engine dtype/expr helpers (pl.DataType / pl.Expr / ExprNode instead of Any); removed verified-redundant cast()s. (The systemic Plottable._nodes: Any root — ~517 casts repo-wide — is filed separately as Type Plottable._nodes/_edges (Any → Optional[DataFrameT]) + make DataFrameT a real polymorphic type #1678, out of scope here.)
Housekeeping: degree helpers moved out of chain.py into degrees.py; bin/test-polars.sh de-duplicated (one test-file list, coverage via env toggle).
Off-engine call() analytics under Polars — call_mode='auto' (default) / 'strict': a call() running a whole-graph analytic with no Polars kernel (umap, hypergraph, compute_cugraph/compute_igraph, *_layout, collapse, …) previously hard-raised NotImplementedError under a Polars engine — forcing users off Polars for the whole pipeline. It now runs as a mode-gated, warned modality switch: auto bridges off-engine (polars→pandas, polars-gpu→cuDF on-device), runs the analytic, and coerces the result back to Polars losslessly via Arrow (warn once per analytic); strict keeps the parity-or-NotImplementedError decline (benchmark integrity / memory ceiling). polars-gpu is GPU-or-error (declines rather than silently dropping to host). Traversal / filter / row ops stay parity-or-NIE — the split is mechanical (is_row_pipeline_call), and the chain and DAG surfaces bridge consistently. Python-settable (set_call_mode) + env GFQL_POLARS_CALL_MODE. dgx-verified: compute_cugraph PageRank byte-parity with the pandas oracle under both polars and polars-gpu; full polars CPU lane green. Documented in the engine guide (docs PR).
More typing (review batch): typed lazy/engine/polars/predicates.py (pl.Expr/pl.DataType + a documented scalar union; audited every is/getattr); a RowPipelineCtx Protocol replaces ctx: Any in row/frame_ops.py (checked interface, zero runtime); explicit __all__ on the lazy public surface.

(Supersedes the former PR2 #1649, folded in here.)

🤖 Generated with Claude Code

Review notes

History: many conventional commits across the engine + folded feat(gfql): polars-gpu = GPU target of the lazy Polars engine (collect-once) [redo of #1654] #1655/feat(gfql/polars): engine followups — native multi-hop, to_fixed_point, undirected, min_hops>1, more predicates + NIE→native #1667 + review-driven refinements (repo lands merge commits); an early-PR module rename (engine_polars/ → lazy/engine/polars/) means the first commits touch pre-rename paths — review by diff, not commit-by-commit.
Team-polish pass applied (commit fe8218c5): stale rename-era docstring paths fixed, internal plan/NO-CHEATING citations scrubbed from user-facing error strings ("no pandas fallback; parity-or-error by design"), Engine.py error strings truthful about polars support, inline engine tuples → POLARS_ENGINES.
Follow-ups (non-blocking, tracked): the repo-wide Plottable._nodes typing (Type Plottable._nodes/_edges (Any → Optional[DataFrameT]) + make DataFrameT a real polymorphic type #1678); unify the 3 mixed-type coercion impls under one protocol-aware helper.
Cleanup pass (2026-07-04, commits 4db768a9..33908cde) — zero-behavior refactor, verified: consolidated the previously-tracked duplication (shared endpoint_ids()/col_dtype() in dtypes.py, one _project_polars body for select/with_columns, _endpoint_counts in degrees, _alias_true_mask in frame_ops, comparison-op whitelists replacing 6-way operator ladders, degree-dispatch collapse) and re-encoded the test suite without shrinking coverage (shared polars_test_utils.py; conformance-ledger axes and the matrix's native-ok/honest-NIE families are table-driven with per-case ids; one chain-parity body with per-table strictness flags; +1 pandas-oracle canary; the chain-vs-DAG surface check is now STRICTER — a non-NIE error on either surface fails). Evidence: dgx full lane + GPU parity 1703 passed at every batch (collected conformance cases 231→232, none lost); polarity/waiver mutation checks fail loudly; ruff+mypy clean. Diff 8,515→8,220 insertions. A follow-up owner-directed prose pass (commit f862657e) tightened the ~1,745 comment/docstring lines (NO-CHEATING contract once per module, per-site one-liners, every fact/citation/perf number kept) — mechanically gated as code-identical (AST-equal with docstrings stripped) + the same dgx lane (1703 passed). Final diff: 7,747 insertions (prod ~3,850 / tests 3,361 / benchmarks+infra 534). A further experiment-gated refactor (commit b956d115) unified the lazy single-hop into hop_polars — ONE hop implementation instead of the twin hop.py/hop_eager.py pair — proven by a 3-round interleaved A/B bench at 1M+10M edges (within-noise-or-better on all workloads; the lazy plan boundary is a measured perf contract) + full parity lane (1703) + the feat(gfql): physical adjacency indexes for O(degree) seeded traversal #1658 index-hook stack re-verified (1724+68). Follow-up typing pass (7acdd411) fully annotates the hop helpers (surfacing and fixing a latent lazy-vs-eager variable name collision). CHANGELOG: unchanged by design — the cleanup/unification is zero-behavior, and the existing feature entries (incl. the collect-once single-hop + eager multi-hop design they describe) remain accurate.

…tructured returns Squashed reconciliation of the native lazy Polars GFQL engine (was #1648's 28 commits; full history preserved at tag bak/1648) restacked onto the colleague's Engine: native polars hop/chain (semi/anti joins), native cypher row pipeline (select/where/order_by/group_by/unwind/projection), lazy single-hop collect-once with CPU/GPU execution targets (gfql/lazy/). NO pandas bridge — native or honest NotImplementedError (plan.md NO-CHEATING). Reconciliation with #1650 structured returns: apply_result_projection now threads `structured` to the polars path (apply_result_projection_polars). Whole-entity RETURN a flattens to {alias}.{field} columns natively (mirrors the pandas _flat_entity_field_names selection exactly), which — unlike the legacy entity-text expr — works for ANY dtype (float/temporal/nested just become columns), so polars structured == pandas structured across the board. structured=False still renders the native Cypher display string for int/string/bool single-entity nodes. _include_numeric_id_as_property is now polars-aware so id flattens identically. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@10m

…alized hops Build the whole forward/backward combine (combine_nodes/edges + endpoint + alias names) as ONE deferred pl.LazyFrame plan over the already-materialized hop frames and collect once, instead of ~a dozen eager ops that each internally lazy().op().collect(). Stable order columns (NORD/EORD) restore the eager g._nodes/g._edges order since lazy joins don't preserve it -> trailing LIMIT/SKIP unaffected, byte-identical (full polars conformance + row-pipeline parity, 2858 gfql tests). NO recompute (inputs materialized; unlike the disproven whole-chain fusion). ~5% faster polars 1-hop chain @1M/@10m; GPU-target neutral. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@1m

A single MATCH (n) with no edge hop — the dominant tabular/crossfilter shape (MATCH (n) WHERE/RETURN ..., histograms, filters, table search) — now returns the filtered node table directly and skips the whole forward/backward/combine + collect_all (~2.5 ms fixed cost that dominated small/interactive queries). Byte-identical (full polars conformance + row-pipeline parity, 389 polars tests). Moves the polars>pandas crossover BELOW 100K for real product workloads: categorical histogram 0.68->1.70x @100k / 1.38->7.62x @1m; node filter 2.44->13.85x @1m; timeline 2.55->8.12x @1m. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@1m

A single MATCH (a)-[e]->(b) with both nodes unconstrained (no filter/name/query) and a plain edge (no match/name/query) — the basic graph query and the viz edge-crossfilter MATCH — returns ALL edges + their endpoint nodes directly (direction-independent; isolated nodes excluded), skipping forward/backward/ combine. For unconstrained nodes the backward pass prunes nothing, so this is byte-identical (full polars conformance + row-pipeline parity + adversarial graphs: dup/self-loop/cycle/isolated). ~9x faster polars [n,e,n]: 95.6->10.3 ms @1m, 855->99 ms @10m. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Extend the unconstrained 1-hop fast path to filtered nodes: MATCH (a {f})-[e]->(b) (src/dst/both filters; the dominant "filter then expand" viz crossfilter pattern) returns the edges whose endpoints pass the node filters + those endpoint nodes, skipping forward/backward/combine. For one hop the backward pass prunes nothing beyond the endpoint filters, so byte-identical (verified vs pandas: src/dst/both filters, reverse, dup/self-loop/cycle/isolated; full polars conformance + row-pipeline parity, 2858 gfql tests). Unconstrained: all edges any direction; filtered: forward/reverse (filtered-undirected falls through to the full path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@10m

…aster) The GPU target collects with pl.GPUEngine(executor="in-memory") instead of the default streaming engine="gpu" (DefaultSingletonEngine). GFQL results fit in device memory, the in-memory engine's regime: faster on the hop primitives (semijoin 1.33x, antijoin 2.58x, unique 1.49x @10m) and far more STABLE -- the streaming executor spiked bimodally to ~1s on the same semijoin (median ~360ms), in-memory holds ~30ms. Fixes the GPU instability seen in the pr11 measurements. Parity preserved (polars-gpu == polars, 39 tests). gfql chains aren't GPU-compute-bound (orchestration + eager fast paths dominate) so this is a stability/correctness fix for GPU-collect paths, not a chain speedup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Per the #1656 author's handoff: the elif-structured single-column text fallback in _apply_result_projection_pandas looks redundant but fixes two regressions (top-level OPTIONAL-MATCH miss; OPTIONAL-WITH-reentry no-match). Mark DO NOT REMOVE so a later 'tidy' doesn't reintroduce them. Our polars structured-returns reconciliation touched this file; verified the fallback is preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…pandas drop_duplicates) RETURN ... UNION RETURN ... (distinct) crashed under engine='polars'/'polars-gpu' with AttributeError: 'DataFrame' object has no attribute 'drop_duplicates' — the union de-dup in gfql_unified._execute_compiled_query called pandas-only drop_duplicates on a polars frame. Added engine-aware Engine.df_unique (polars unique(maintain_order=True); pandas/cuDF drop_duplicates(keep='first')), matching the row/frame_ops.distinct convention, and routed the UNION DISTINCT through it. Surfaced by the cross-repo TCK conformance run (tck-gfql TEST_POLARS=1, union1). Regression-tested in test_engine_polars_cypher_conformance.py (4 UNION cases). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…t to Boolean) null AND null / null OR null / NOT null crashed under engine='polars' with InvalidOperationError ('bitand'/'not' not supported for dtype null): a bare null literal lowers to a Null-dtype polars expr where &/|/~ are undefined. Cast AND/OR/ NOT operands to pl.Boolean in the expr lowering so Cypher Kleene 3-valued logic evaluates (true AND null=null, false OR null=null, NOT null=null); casting a real Boolean column is a no-op, and polars Boolean &/|/~ already match Cypher Kleene. Surfaced by the TCK run (expr-boolean1/2/4). Regression-tested in test_engine_polars_cypher_conformance.py (bare-RETURN null boolean cases). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ot ArrowInvalid A pandas object column holding mixed Python types (e.g. int 0 + str 'xx' — legal for dynamically-typed Cypher properties) is unrepresentable in polars/Arrow: pl.from_pandas raised a cryptic 'pyarrow.lib.ArrowInvalid: Could not convert xx with type str: tried to convert to int64' from deep inside construction. Wrap the pandas->polars conversion in Engine.df_to_engine (_pl_from_pandas) to raise a clear NotImplementedError naming the offending column(s) and pointing at engine='pandas' (NO-CHEATING: no silent string-coercion, which would change comparison semantics). Surfaced by the TCK run (expr-comparison2, match-where5, with-where5). The harness tolerates honest NIE as a coverage decline; before this they crashed as failures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…projection A property column holding Cypher temporal-constructor text (date({year:1910,...}), how Cypher/TCK store temporal values) leaked the raw constructor string under engine='polars' instead of the ISO form ('1910-05-06') the pandas projection produces via _normalize_temporal_constructor_series. That normalizer is not yet native, so both projection paths (engine_polars.projection final result projection + row_pipeline.select_polars WITH/RETURN) now detect temporal-constructor String columns (reusing TEMPORAL_CALL_EXPR_RE, native .str.contains scan over String cols only) and raise NotImplementedError rather than emit a wrong rendering. Surfaced by the TCK run (with-orderby1-33+, the largest wrong-answer cluster, ~33). Whole-entity RETURN a over a temporal property is unaffected (flattens + renders via render_entity_text). Regression-tested in test_engine_polars_cypher_conformance.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… +/-) a.time + duration({minutes: 6}) silently became STRING CONCATENATION under engine='polars': cypher duration({...}) translates to an ISO duration string literal ('PT6M'), and the expr lowering applied + to two strings, so an ORDER BY sorted lexicographically on the concatenated text (wrong order). The lowering now raises NotImplementedError when +/- has an ISO-duration string-literal operand (^-?P(?=[0-9T]), which doesn't misfire on ordinary strings like 'Prefix'); the pandas engine handles temporal arithmetic. Surfaced by the TCK run (with-orderby2 cluster, silent wrong-order). Regression- tested in test_engine_polars_cypher_conformance.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…n native lowering filter_by_dict on engine='polars' evaluated any non-natively-lowerable predicate by converting the column to pandas (.to_pandas()), running the pandas callable, and carrying the mask back — a silent polars->pandas bridge presenting pandas semantics as polars. Removed it: unsupported predicates now raise NotImplementedError (use engine='pandas'). To keep common queries native, widened predicate_to_expr: - AllOf (conjunction, e.g. n.val > 20 AND n.val < 90 -> AllOf[GT,LT]) lowered recursively - IsNull/IsNA -> is_null(), NotNull/NotNA -> is_not_null() - case-insensitive STARTS WITH / ENDS WITH via anchored (?i) regex on re.escape'd literal Surfaced from the source-mined optimization review (pygraphistry4 opportunity #6 — a flagged NO-CHEATING violation in the shipping polars lane). The old fallback test (which asserted the bridge worked) now asserts the honest NIE. TCK: no wrong-answer regression; ~39 scenarios that silently passed via the bridge now honestly decline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

….iloc crash) A bounded MATCH ... WITH <scalar> ... MATCH query crashed under engine='polars' with AttributeError: 'DataFrame' object has no attribute 'iloc' — the engine- agnostic re-entry broadcast (cypher/reentry/execution.py) used pandas .iloc / .assign / .drop(columns=) on a polars frame. Added engine-aware helpers (polars row(i, named=True) + with_columns(pl.lit(...)) / drop / head(0)) for the scalar-row extraction + constant-column broadcast. Re-entry now completes; a downstream RETURN the polars engine can't yet render raises honest NotImplementedError, not a crash. Surfaced by the TCK run (with2-1, with4-2, expr-typeconversion2/3/4-*). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ot validation error) A multi-clause OPTIONAL MATCH needing null-row fill (some seed rows unmatched) raised GFQLValidationError ('unsupported-cypher-query ... null-row alignment could not recover matched seed identities') under engine='polars' — the null-fill alignment (matched-id meta, .iloc row slicing, per-segment concat) is pandas-centric and the polars OPTIONAL MATCH doesn't populate the _cypher_entity_projection_meta ['ids'] it needs. Guarded the polars path to raise NotImplementedError instead (the honest 'not native yet' signal the TCK harness tolerates) — pandas runs these fine. Surfaced by the TCK run (match7-7, expr-graph4-4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

OPTIONAL MATCH (n) RETURN n with no match rendered the absent whole entity as '()' under engine='polars' instead of null — the native entity-text expr didn't nullify absent rows (whose alias marker column is null). Now wraps the rendered text with pl.when(col(alias).is_null()).then(None) (mirrors pandas _nullify_missing_alias_rows); a real property-less node still renders '()'. Surfaced by the TCK run (match7-1). Regression-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… (not == cast crash) A label match MATCH (n:Label) targeting the reserved 'labels' List column (a label with no one-hot label__X column: typed-schema unknown labels, OPTIONAL MATCH to a non-existent label) crashed under engine='polars' with InvalidOperationError: cannot cast List type to String — filter_by_dict_polars lowered it to a scalar == that tried to cast the List to String. Now uses pl.col(c).list.contains(val) for List-dtype columns: correct Cypher label-membership (Label in n.labels), empty for a non-existent label (matching pandas). Surfaced by the TCK run (match7-28, firstparty-typed-schema1-3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

NaN: comparisons over a NaN computed inside polars (0.0/0.0 > 1) used polars' semantics (NaN = largest value, NaN>1 True), but IEEE/Python/pandas/Neo4j-Cypher compare any NaN false (!= true). The expr lowering now masks float comparisons to the IEEE answer (& ~is_nan for < > <= >= =, | is_nan for <> !=), gated by conservative float-operand inference (via a free schema contextvar) so int/string/ bool comparisons are untouched and is_nan() never hits a non-float expr. Input NaN is already nan_to_null'd by pl.from_pandas, so this only affects in-query float math. Numeric-vs-string: comparing a number to a string (n.val > 'a', 0.0/0.0 > 'a') crashed with ComputeError: cannot compare string with numeric type. Detect the mismatch in both the expression path (lower_expr) and the folded filter-predicate path (filter_by_dict_polars) and raise honest NotImplementedError, not a crash. Surfaced by the TCK run (expr-comparison2-5-*, the 4-scenario NaN cluster). Regression-tested in test_engine_polars_cypher_conformance.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…aphic wrong-answer) Comparing cypher temporal values (time({...}) > time({...}), date < date) gave a WRONG answer under engine='polars': the cypher->gfql lowering renders the constructors to ISO strings ('10:00+01:00'), and the polars engine compared them LEXICOGRAPHICALLY — wrong across timezones/precision (pandas parses them temporally). The lowering now detects an ISO date/datetime/time string-literal operand in a comparison (specific regex; requires seconds-or-tz on bare times so ordinary '10:00' strings don't match) and raises honest NotImplementedError. Native temporal-typed comparison is the tracked proper fix. With this, the native polars engine has ZERO wrong-answers across the full Cypher TCK (3834 passed / 0 failed / 388 honest declines) — every scenario matches pandas or honestly declines. Surfaced by the TCK run (expr-temporal7). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…oral guards Multi-wave adversarial review of the session conformance fixes found 3 BLOCKERs (silent wrong-answers/panics, all NO-CHEATING violations) + IMPORTANTs: - BLOCKER: NaN guard missed int/int->Float division and function results (abs/ coalesce) -> polars NaN-as-largest leaked as wrong answers. Now drive the NaN + cross-type guards from the lowered exprs OUTPUT dtype (_expr_output_dtype, schema- only) instead of AST type inference — robustly catches division/functions. Replaces the three _infer_is_* helpers (DRY). - BLOCKER: list.contains was applied to ANY List column, so a user List property (n.tags = scalar) returned membership (wrong) vs pandas equality. Gated to the reserved labels column; other List columns decline honestly. - BLOCKER: numeric-vs-string nested in AllOf (x>20 AND x<z) or Between bypassed the cross-type guard and PANICKED (uncatchable Rust). _is_cross_type_predicate now recurses AllOf/Between. - IMPORTANT: Categorical/Enum columns now treated as string-like in both cross-type guards (categorical-vs-numeric was a raw ComputeError). - IMPORTANT: all-null columns (typed String by from_pandas) crashed on arithmetic (n.val + 1); cross-type guard now covers arithmetic ops, not just comparison. - IMPORTANT: ISO-temporal comparison guard narrowed to ORDERING of two temporal literals (was declining valid string-column-vs-date-literal compares; = and <> are lexicographically correct so not declined). - Anchored the temporal-constructor scan regex (no false-positive on update(...)). - Added the missing CHANGELOG entry (OPTIONAL MATCH null-fill decline) + streaming comment clarity. Full TCK still 3834 passed / 0 wrong-answers / 387 honest declines. 457 polars tests pass. 6 new adversarial regression tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… OTel span placement Adversarial review of the lower stack layers (#1648 polars engine, #1652 generic fast paths) found 2 bugs: - BLOCKER (#1648): a chain crashed under engine=polars with SchemaError when an edge endpoint dtype differed from the node-id dtype across int<->float (e.g. a null in a source/dest column -> float64 vs int64 ids) where pandas joins fine. The hop aligns join keys; the chain fast paths + combine did not. Added _align_edge_endpoints (cast endpoints to node-id dtype for the traversal, restore output dtype to match pandas; no-op when dtypes match) wired into the single-hop fast path + multi-hop. - (#1652): the gfql.chain OTel @otel_traced decorator had landed on the internal _try_chain_fast_path probe (inserted between the decorator and def chain) instead of the public chain() — chain() lost its span, span recorded wrong fn/attrs. Moved it. Both verified + regression-tested. 457 polars tests + 334 generic chain tests pass (4 fails are the pre-existing local libnvrtc CUDA-env issue, not these changes). Row-order divergences the review also found (fast path returns table order vs the full machinery BFS-discovery order, for reverse/undirected without ORDER BY — Cypher- undefined, sets/values identical, no repo test depends on it) are claim-precision, not bugs; CHANGELOG wording to be tightened in the per-layer #1648/#1652 review pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ype + DRY/docs Multi-dimension adversarial review (correctness/robustness/quality/docs) of the native polars engine found three reachable pandas-oracle divergences, now fixed (NO-CHEATING — match pandas or decline honestly): - BLOCKER: duplicate alias [n('a'), e(), n('a')] returned a malformed colliding- join schema (a/a_right) instead of raising; now raises GFQLValidationError E201 like pandas (node/edge aliases scoped separately, mirroring combine_steps). - BLOCKER: integer-literal division 5/2 lowered to polars true division (2.5) but Cypher folds to int division (2) — silent wrong order when embedded non- monotonically (ORDER BY n.val % (10/4)); now declines (NIE). Column / int (Float on both) unaffected. - IMPORTANT: internal start_nodes seed with a divergent id dtype (empty crossfilter -> float64 vs int64 node ids) crashed the combine join (SchemaError); now aligns the seed key (_align_seed_dtype), mirroring the hop + edge-endpoint alignment. Quality/docs: - Removed stale 'pandas bridge' docstrings/comments (row_pipeline, projection) — the bridge was removed in the de-cheat commit; the code raises NIE. - DRY: consolidated the cross-type/NaN dtype classifiers (numeric/int/float/ stringlike), duplicated 4x, into engine_polars/dtypes.py (the guard contract). - Aligned the lazy hop allowed_source guard textually with the eager hop (no-op: to_fixed_point is NIE'd upstream) to stop future eager/lazy drift. - Removed two dead imports in chain.py (hop_polars, Engine). - Documented a narrow filter_by_dict genuine-NaN residual (unreachable on the from_pandas ingestion path that null's NaN). +3 regression tests. 457 polars+chain tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ion (NaN vs None) CI polars 1.40.1 renders a null entity column as NaN (newer polars: None) on polars->pandas conversion. Assert is-null (pd.isna) instead of == [None]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…rs/ (was engine_polars/) Pure move, no logic change. lazy/__init__ already specifies per-backend lowering under lazy/engine/<backend>/ (future lazy/engine/duckdb, .../dask); the eager polars lowering was in a flat sibling engine_polars/. Collapse it into the one polars-backend home so future lazy backends don't proliferate flat engine_* siblings: - engine_polars/{chain,dtypes,predicates,projection,row_pipeline}.py -> lazy/engine/polars/ - engine_polars/hop.py (eager) -> lazy/engine/polars/hop_eager.py (distinct name; the existing lazy collect-once hop.py is left in place — no collision). - engine_polars/__init__ re-exports folded into lazy/engine/polars/__init__; engine_polars/ removed. - imports rewritten engine_polars -> lazy.engine.polars (eager hop -> .hop_eager). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@1m

Redo of the per-op GPU engine (#1654, a perf regression) as a TARGET of the lazy engine: engine='polars-gpu' runs the same single-deferred-plan + collect-once on the cudf_polars GPU backend. Tiny wiring on top of the lazy engine — the lazy/ framework already does target-aware collect. - Engine.POLARS_GPU = 'polars-gpu' + POLARS_ENGINES; explicit opt-in (AUTO never picks it); frames stay pl.DataFrame (treated like POLARS in frame ops). - compute/{hop,chain}.py dispatch: engine in (POLARS, POLARS_GPU) -> wrap the lazy call in target_mode(GPU if POLARS_GPU else CPU). ComputeMixin + gfql_unified same-path WHERE accept POLARS_GPU. engine='polars' (CPU) byte-for-byte unchanged. - raise_on_fail=False (GPU-incapable nodes stay on CPU in polars; no pandas bridge). dgx: parity engine='polars-gpu' == engine='polars' (test_engine_polars_gpu.py 36 passed); full gfql suite 2921 passed, 0 failed. Single-hop GPU 2.84x @1m (vs the per-op regression); chain-level GPU win currently dilutes (fwd+bwd 2 collects + eager combine) -> next opt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…(P-B) GFQL_POLARS_CPU_STREAMING=1 runs the polars-CPU lazy collects (hop/chain) on the streaming executor. Benchmarked (dgx, interleaved A/B, parity-identical): ~1.11x at 10M nodes/80M edges (20.0->18.0s), ~1.04x at 1M, but ~0.86x (slower) at 100K — the streaming overhead loses on small/interactive sizes. So default OFF (behavior unchanged); opt-in for large batch traversals. From the blogpost perf-opt handoff item B (polars-CPU heavy-join scaling). The full streaming win in isolation is larger (80M 2-hop semijoin 1669->1040ms, 1.6x); the real chain dilutes it via the forward/backward/combine overhead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ragma GPU collect #1655 changed-line-coverage gate (newly enforced once upstream jobs pass) flagged 8 GPU/polars dispatch lines. Fix honestly: a CPU test exercises the lazy collect()/ collect_all() CPU path + the POLARS branches of df_concat/df_cons/s_cons/df_to_engine (reachable but not hit by the coverage suites); only the 2 genuinely GPU-target collect lines (lf.collect(engine=gpu) / pl.collect_all(..., engine=gpu)) are pragma:no-cover (need a device CI lacks). Changed-line coverage of #1655 back to ~100%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…allback (NO-CHEATING) The lazy GPU collect used pl.GPUEngine(raise_on_fail=False): any plan node the cudf_polars backend cannot execute silently ran on CPU and was still reported as a polars-gpu result -- so engine='polars-gpu' was indistinguishable from engine='polars' whenever the plan was not fully GPU-capable. A bulk bench showing near-identical polars/polars-gpu timings is exactly this tell. Flip to raise_on_fail=True and translate the cudf_polars failure into a clear NotImplementedError pointing at engine='polars'. polars-gpu is now GPU-or-error: any timing it produces is real on-device work, never CPU mislabeled as GPU. Verified on dgx-spark (LiveJournal 35M): the seeded hop / 2-hop chain plan runs fully on GPU without raising (nvidia-smi 92% util), so existing GPU timings are unchanged -- only the honesty guarantee is added. +1 regression test covering the translated error path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…to' honesty Add test_gpu_executor_modes_parity[in-memory|streaming]: runs a traversal under each cudf-polars GPU executor on a real GPU and asserts parity with CPU polars (pandas-gated). Locks in the 'streaming' executor, previously covered only by a mock-wiring assertion + manual dgx runs. Skipped in CI (no GPU); runs on the dgx GPU lane. dgx-verified: 2 pass. Also tighten the gpu_executor() docstring: 'in-memory'/'streaming' are the ONLY selectable values; a size-aware 'auto' is a possible future addition, NOT selectable today (set_gpu_executor('auto') raises) — so a reader doesn't mistake it for a current option. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…r ladders (cleanup P1) pl.Expr implements the Python operator protocol, so op(lhs, rhs) builds the identical expression the if-ladders did. predicates._cmp_expr gains _CMP_OPS (both the DateValue and scalar ladders collapse); row_pipeline._apply_binop gains _BINOP_FNS. No behavior change; parity comments retained. Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ort, drop dead can_* probes (cleanup P2) select_polars and with_columns_polars were byte-identical except for the projection method — shared _project_polars(extend=) keeps ONE copy of the temporal-constructor NO-CHEATING decline. _lower_function's 7 branch-local 'import polars as pl' hoist to one function-level import (polars stays an optional dep). can_select_native/can_order_by_native had zero callers repo-wide (grep-verified) — removed. Module docstring corrected: CaseWhen and the function whitelist ARE lowered (was stale). No behavior change. Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…at copies (cleanup P3) The src-stacked-on-dst node-id-universe concat (with the int/float join-key dtype cast polars won't do implicitly) appeared verbatim 8x across hop/hop_eager/chain. One dtypes.py helper builds the identical expression; each call site keeps its own .unique(...) variant (plain vs subset= is load-bearing for lazy maintain_order). Typed with a constrained PolarsT TypeVar (eager-in/eager-out, lazy-in/lazy-out). Built plans unchanged. Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…nts/col_dtype (cleanup P4) executor.py's three per-function import+call blocks collapse to a membership test over the explicit (grep-able) name tuple + getattr on the degrees module. degrees.py's three group_by-cast-count expressions share _endpoint_counts (the int/float join-key cast comment lives once); node-dtype lookups use the new dtypes.col_dtype. chain._try_native_row_op degree branches intentionally kept unrolled (collapsing costs legibility for ~1 line). No behavior change. Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…leanup P5) rows() and count_table() carried two divergent copies of the alias-mask NULL->False logic (where/isna vs plain fillna — same intent); one helper now serves both (rows' more defensive variant; identical result, 4-engine count_table parity covers it). ComputeMixin's two inline polars module-string checks swap to Engine.is_polars_df (the declared single source of truth). Skipped as not-worth-it: resolve_engine swap (polars-Series edge case), and/or reduce()-folds (3 idiomatic lines each, reduce is not clearer), chain degree-branch collapse (legibility loss for ~1 line). Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…-> AXES table + 7 parametrized checks (cleanup T1) Same 28 test cases (7 checks x 4 axes, per-axis parametrize ids), same failure messages (registry/exercised/waiver names injected), all four waiver dicts byte-identical — the ledger CONTENT is untouched, only its test encoding is table-driven. A new coverage axis is now one AXES entry. Mutation-verified: deleting the AllOf waiver fails [predicates] naming AllOf; a bogus waiver key fails the stale-waiver check on both axes it lands in. Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…hared polars_test_utils (cleanup T2+T3a) polars_test_utils.py now holds the ONE definition of the comparison machinery (to_pandas_any / typed_frame_sig / run_status / available_nonpandas_engines / assert_parity_or_nie / assert_surfaces_agree) with loud-failure contracts documented. The matrix's hand-unrolled clone families become tables that keep per-case ids + one-line 'why' rationales: _NATIVE_OK_CYPHER (7) + _HONEST_NIE_CYPHER (5) + _native_ok_query_cases (4) + _polars_nie_query_cases (10); the degree trio unifies over one fn parametrize (native probe kept for all 3). assert_surfaces_agree STRENGTHENS the old inline chain-vs-dag check (a non-NIE 'err' on either surface now fails; before it fell through). Added test_pandas_oracle_sanity canary (a global oracle break would otherwise silently skip the matrix). Collected cases: 231 -> 232 (+1 canary, zero subjects lost); ledger anchors (_predicate_queries etc.) untouched. dgx full lane + GPU: 1703 passed. Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… one chain-parity body (cleanup T3b+T4) hop's _node_set/_edge_set/_node_attrs_hop were verbatim/behavior-identical copies of chain's _nset/_eset/_node_attrs — all five graph-shape helpers (node_id_set/edge_pair_set/edge_pair_multiset/node_attr_map/named_flag_set) now live once in polars_test_utils (no try/except, no non-empty defaults — loud-failure contract documented), aliased at import so call sites read unchanged. The five near-identical chain parity bodies (CHAINS, both fuzzers, adversarial multihop, to_fixed_point, undirected-multiedge) collapse into _assert_chain_parity with per-table opt-in flags (multiplicity/attrs = the min_hops-bug dimensions; native probe; edge-count; aliases; nie_skip reason). Same checks per table as before — encoding only. dgx full lane + GPU: 1703 passed (same count). Part of the #1660 cleanup pass (plans/gfql-engine-followups/cleanup-1660). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…s (no crash) GPU-parity pass (viz-filter #1673 item 2) on dgx found `MATCH (n) WHERE n.name =~ '(?i)…'` CRASHES on engine='cudf' with libcudf "invalid regex pattern: nothing to repeat". Root cause: libcudf's regex engine rejects inline flag groups ((?i)/(?m)/ (?s)) at ANY position (verified: '(?i)abc', '^(?i)abc$', '(?i)^abc$' all raise; only flag-free '^abc$' works) — not merely a position issue. Fix: the Match/Fullmatch cuDF branches now translate a leading (?i) to the existing lowercase case-folding workaround (parity with pandas' (?i)), and honestly decline any other inline flag with NotImplementedError instead of crashing. Shared helper _cudf_regex_prep. pandas/polars paths untouched. Validated on dgx (RAPIDS 26.02): cudf =~ '(?i)a.c' == pandas [2,3] (was RuntimeError); 446 regex/match/fullmatch/contains/numeric tests pass across pandas/polars/cudf; +1 cudf-gated regression test (test_regex_cudf_inline_flag_parity). ruff+mypy clean. Also confirms viz-filter #1673 item 2: cuDF numeric (floor/ceil/round) + toLower/ toUpper parity OK; polars-gpu is N/A on this branch (#1675 is off #1660, below #1655 which introduces polars-gpu). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…docstring, NB placement (cleanup review) Adversarial-review outcome on the cleanup diff (aa8c3e5..33908cd): behavior + coverage preserved, zero blockers. Nits fixed: the '%' dict comment no longer claims a parity verification no conformance case backs (negative- operand % case tracked as follow-up); endpoint_ids docstring no longer overstates plain-vs-subset unique as load-bearing on its one-column output (kept per-site for byte-identical diffs); the _polars_nie_query_cases NB comment moved from after the return into the docstring. Comment-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ceil/ceiling/round/toLower/toUpper) The coverage ledger (cascaded from #1660) correctly flagged the new GFQL_SCALAR_FUNCTIONS entries as neither exercised nor waived. Real 4-engine parity-or-NIE cases beat waivers: each new function gets a cypher expression case through the standard matrix driver (chain + DAG surfaces). dgx: matrix + ledger 267 passed (cudf + polars-gpu lanes active). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…e change (cleanup P8+T6) Owner-directed rewrite of the ~1,745 comment/docstring lines across the 9 engine modules + 9 polars test files: the NO-CHEATING/parity-or-NIE contract now states once per module docstring with short per-site 'decline (NIE): <reason>' forms; multi-line parity narratives compressed ~35-50% keeping every distinct fact (all fuzz-seed citations, pandas source line refs, perf numbers, dtype gates, divergence counterexamples; public-API docstrings keep every accepted value/default/env var/precedence rule). Ledger waiver dicts and case- table reason strings untouched (data, not prose). Mechanically gated: every file AST-identical to HEAD with docstrings stripped (prose_check), independently re-verified; ruff+mypy clean; dgx full lane + GPU parity 1703 passed (same count). -384 lines (prod -205, tests -179). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…e hop implementation (cleanup A1) The single-bounded-hop lazy collect-once plan (formerly hop.py's 176-line twin, kept 'textually identical' by discipline) is now an early branch inside hop_polars, sharing the setup/validation/epilogue with the eager loop; hop.py collapses to the thin hop_lazy_or_eager entry (stable hook site). Eager loop math untouched. The branch's seed/match/target id-frames stay INSIDE the lazy plan — A/B benching (3 interleaved rounds, 1M+10M edges, dgx) proved that boundary is a perf contract: eager gate materialization cost +5-14% on chain workloads. Final round: within-noise-or-better on all 10 workloads (hop1 -7..9%, chain 2-edge -4%); full parity lane + GPU 1703 passed. Experiment history (2 gate-caught bugs incl. a return_as_wave_front wrong-answer) in plans/gfql-engine-followups/cleanup-1660/plan.md Step 9.A1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…s (no crash) GPU-parity pass (viz-filter #1673 item 2) on dgx found `MATCH (n) WHERE n.name =~ '(?i)…'` CRASHES on engine='cudf' with libcudf "invalid regex pattern: nothing to repeat". Root cause: libcudf's regex engine rejects inline flag groups ((?i)/(?m)/ (?s)) at ANY position (verified: '(?i)abc', '^(?i)abc$', '(?i)^abc$' all raise; only flag-free '^abc$' works) — not merely a position issue. Fix: the Match/Fullmatch cuDF branches now translate a leading (?i) to the existing lowercase case-folding workaround (parity with pandas' (?i)), and honestly decline any other inline flag with NotImplementedError instead of crashing. Shared helper _cudf_regex_prep. pandas/polars paths untouched. Validated on dgx (RAPIDS 26.02): cudf =~ '(?i)a.c' == pandas [2,3] (was RuntimeError); 446 regex/match/fullmatch/contains/numeric tests pass across pandas/polars/cudf; +1 cudf-gated regression test (test_regex_cudf_inline_flag_parity). ruff+mypy clean. Also confirms viz-filter #1673 item 2: cuDF numeric (floor/ceil/round) + toLower/ toUpper parity OK; polars-gpu is N/A on this branch (#1675 is off #1660, below #1655 which introduces polars-gpu). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ceil/ceiling/round/toLower/toUpper) The coverage ledger (cascaded from #1660) correctly flagged the new GFQL_SCALAR_FUNCTIONS entries as neither exercised nor waived. Real 4-engine parity-or-NIE cases beat waivers: each new function gets a cypher expression case through the standard matrix driver (chain + DAG surfaces). dgx: matrix + ledger 267 passed (cudf + polars-gpu lanes active). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

_hop_setup_columns/_build_hop_pairs/_min_hops_labeled_node_output and the _idframe closures get full signatures via the TYPE_CHECKING-polars pattern (PolarsT for the eager/lazy-generic pairs builder). Typing the defs made mypy check the whole body, which surfaced a function-wide name collision: the single-shot lazy branch bound hop_edges as LazyFrame — renamed to hop_edges_lf (consistent with the branch's _lf convention). ruff+mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…hop-entry hook) The polars lane's per-file coverage audit gates hop.py; after #1660's hop unification the file is a thin entry dominated by this PR's index hook, which the lane didn't exercise (57% < 87% floor). Adding the engine-parametrized index tests to POLARS_TEST_FILES exercises the hook for real: dgx-measured 92.86% (>87 floor), 1724 lane tests pass. Real coverage, not floor surgery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…s (no crash) GPU-parity pass (viz-filter #1673 item 2) on dgx found `MATCH (n) WHERE n.name =~ '(?i)…'` CRASHES on engine='cudf' with libcudf "invalid regex pattern: nothing to repeat". Root cause: libcudf's regex engine rejects inline flag groups ((?i)/(?m)/ (?s)) at ANY position (verified: '(?i)abc', '^(?i)abc$', '(?i)^abc$' all raise; only flag-free '^abc$' works) — not merely a position issue. Fix: the Match/Fullmatch cuDF branches now translate a leading (?i) to the existing lowercase case-folding workaround (parity with pandas' (?i)), and honestly decline any other inline flag with NotImplementedError instead of crashing. Shared helper _cudf_regex_prep. pandas/polars paths untouched. Validated on dgx (RAPIDS 26.02): cudf =~ '(?i)a.c' == pandas [2,3] (was RuntimeError); 446 regex/match/fullmatch/contains/numeric tests pass across pandas/polars/cudf; +1 cudf-gated regression test (test_regex_cudf_inline_flag_parity). ruff+mypy clean. Also confirms viz-filter #1673 item 2: cuDF numeric (floor/ceil/round) + toLower/ toUpper parity OK; polars-gpu is N/A on this branch (#1675 is off #1660, below #1655 which introduces polars-gpu). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ceil/ceiling/round/toLower/toUpper) The coverage ledger (cascaded from #1660) correctly flagged the new GFQL_SCALAR_FUNCTIONS entries as neither exercised nor waived. Real 4-engine parity-or-NIE cases beat waivers: each new function gets a cypher expression case through the standard matrix driver (chain + DAG surfaces). dgx: matrix + ledger 267 passed (cudf + polars-gpu lanes active). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…s (no crash) GPU-parity pass (viz-filter #1673 item 2) on dgx found `MATCH (n) WHERE n.name =~ '(?i)…'` CRASHES on engine='cudf' with libcudf "invalid regex pattern: nothing to repeat". Root cause: libcudf's regex engine rejects inline flag groups ((?i)/(?m)/ (?s)) at ANY position (verified: '(?i)abc', '^(?i)abc$', '(?i)^abc$' all raise; only flag-free '^abc$' works) — not merely a position issue. Fix: the Match/Fullmatch cuDF branches now translate a leading (?i) to the existing lowercase case-folding workaround (parity with pandas' (?i)), and honestly decline any other inline flag with NotImplementedError instead of crashing. Shared helper _cudf_regex_prep. pandas/polars paths untouched. Validated on dgx (RAPIDS 26.02): cudf =~ '(?i)a.c' == pandas [2,3] (was RuntimeError); 446 regex/match/fullmatch/contains/numeric tests pass across pandas/polars/cudf; +1 cudf-gated regression test (test_regex_cudf_inline_flag_parity). ruff+mypy clean. Also confirms viz-filter #1673 item 2: cuDF numeric (floor/ceil/round) + toLower/ toUpper parity OK; polars-gpu is N/A on this branch (#1675 is off #1660, below #1655 which introduces polars-gpu). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ceil/ceiling/round/toLower/toUpper) The coverage ledger (cascaded from #1660) correctly flagged the new GFQL_SCALAR_FUNCTIONS entries as neither exercised nor waived. Real 4-engine parity-or-NIE cases beat waivers: each new function gets a cypher expression case through the standard matrix driver (chain + DAG surfaces). dgx: matrix + ledger 267 passed (cudf + polars-gpu lanes active). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…s (no crash) GPU-parity pass (viz-filter #1673 item 2) on dgx found `MATCH (n) WHERE n.name =~ '(?i)…'` CRASHES on engine='cudf' with libcudf "invalid regex pattern: nothing to repeat". Root cause: libcudf's regex engine rejects inline flag groups ((?i)/(?m)/ (?s)) at ANY position (verified: '(?i)abc', '^(?i)abc$', '(?i)^abc$' all raise; only flag-free '^abc$' works) — not merely a position issue. Fix: the Match/Fullmatch cuDF branches now translate a leading (?i) to the existing lowercase case-folding workaround (parity with pandas' (?i)), and honestly decline any other inline flag with NotImplementedError instead of crashing. Shared helper _cudf_regex_prep. pandas/polars paths untouched. Validated on dgx (RAPIDS 26.02): cudf =~ '(?i)a.c' == pandas [2,3] (was RuntimeError); 446 regex/match/fullmatch/contains/numeric tests pass across pandas/polars/cudf; +1 cudf-gated regression test (test_regex_cudf_inline_flag_parity). ruff+mypy clean. Also confirms viz-filter #1673 item 2: cuDF numeric (floor/ceil/round) + toLower/ toUpper parity OK; polars-gpu is N/A on this branch (#1675 is off #1660, below #1655 which introduces polars-gpu). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ceil/ceiling/round/toLower/toUpper) The coverage ledger (cascaded from #1660) correctly flagged the new GFQL_SCALAR_FUNCTIONS entries as neither exercised nor waived. Real 4-engine parity-or-NIE cases beat waivers: each new function gets a cypher expression case through the standard matrix driver (chain + DAG surfaces). dgx: matrix + ledger 267 passed (cudf + polars-gpu lanes active). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…hop-entry hook) The polars lane's per-file coverage audit gates hop.py; after #1660's hop unification the file is a thin entry dominated by this PR's index hook, which the lane didn't exercise (57% < 87% floor). Adding the engine-parametrized index tests to POLARS_TEST_FILES exercises the hook for real: dgx-measured 92.86% (>87 floor), 1724 lane tests pass. Real coverage, not floor surgery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…hop-entry hook) The polars lane's per-file coverage audit gates hop.py; after graphistry#1660's hop unification the file is a thin entry dominated by this PR's index hook, which the lane didn't exercise (57% < 87% floor). Adding the engine-parametrized index tests to POLARS_TEST_FILES exercises the hook for real: dgx-measured 92.86% (>87 floor), 1724 lane tests pass. Real coverage, not floor surgery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

lmeyerov mentioned this pull request Jun 29, 2026

feat(gfql): native lazy Polars engine — collect-once traversals + cypher row pipeline #1648

Closed

lmeyerov and others added 28 commits June 30, 2026 18:13

lmeyerov force-pushed the dev/gfql-polars-engine branch from f7dcde6 to 8577f4f Compare July 1, 2026 01:19

lmeyerov and others added 9 commits July 4, 2026 12:03

lmeyerov and others added 2 commits July 4, 2026 14:18

lmeyerov merged commit 4dbe060 into master Jul 4, 2026
79 checks passed

lmeyerov deleted the dev/gfql-polars-engine branch July 4, 2026 23:39

lmeyerov mentioned this pull request Jul 5, 2026

feat(gfql/cypher): standard =~ regex + numeric fns + toLower/toUpper (#1673) #1675

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(gfql): native lazy Polars engine — collect-once traversals + cypher row pipeline + followups (multi-hop, to_fixed_point, undirected, more predicates, NIE→native)#1660

feat(gfql): native lazy Polars engine — collect-once traversals + cypher row pipeline + followups (multi-hop, to_fixed_point, undirected, more predicates, NIE→native)#1660
lmeyerov merged 115 commits into
masterfrom
dev/gfql-polars-engine

lmeyerov commented Jun 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

lmeyerov commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

GPU target — engine='polars-gpu' (folded #1655)

Followups — native completions (folded #1667)

Traversals — hop() / chain()

Cypher row pipeline — MATCH … RETURN

Conformance hardening

Validation

Perf (interleaved, 1M nodes, each engine on its native-frame graph, all native)

Recent refinements (review-driven)

Review notes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lmeyerov commented Jun 29, 2026 •

edited

Loading

GPU target — `engine='polars-gpu'` (folded #1655)

Traversals — `hop()` / `chain()`

Cypher row pipeline — `MATCH … RETURN`