feat(gfql/cypher): searchAny cross-column search predicate — inspector semantics, native on 4 engines#1680
Merged
Merged
Conversation
d50952f to
dc9983b
Compare
dc9983b to
b246e72
Compare
2e7f20f to
c0a8c1f
Compare
ec19d78 to
c0634f1
Compare
lmeyerov
commented
Jul 5, 2026
| from .row_pipeline import _active_table, _rewrap | ||
|
|
||
|
|
||
| def search_any_polars( |
Contributor
Author
There was a problem hiding this comment.
ensure cross-platform not just polars, and cross-test
lmeyerov
commented
Jul 5, 2026
lmeyerov
commented
Jul 5, 2026
| else: | ||
| exprs.append(base.str.to_lowercase().str.contains(term.lower(), literal=True)) | ||
| marked = left.with_columns(pl.any_horizontal(exprs).fill_null(False).alias(out_col)) | ||
| return _rewrap(g, marked) |
Contributor
Author
There was a problem hiding this comment.
we should audit our tests:
- what happens when nested structures
- are we appropriatley short circuiting, eg, skipping bool cols and numeric cols for non-numeric str
- ... what happens when pretty printer adds "." and "," to ui? should we search those too?
- true/false variants (True, False, 0, 1, ...) can probably be converted to bool vals for when searching bool cols, what's normal here?
c0a8c1f to
08c1e01
Compare
c0634f1 to
9af5808
Compare
lmeyerov
commented
Jul 5, 2026
| "search_edges columns= includes a column absent from the edges table", | ||
| field="columns", value=columns, | ||
| suggestion="List only columns present on the edges table.") | ||
| return self.edges(df[mask]) |
08c1e01 to
889f59e
Compare
dbf67ae to
5b692fe
Compare
…-filter L2-a) Kernel (gfql/search_any.py): OR-across-columns, case-insensitive substring default, regex opt-in, dtype gate AS SEMANTICS (string cols always; int cols iff numeric- literal term per the inspector /^[0-9.-]+$/ gate; floats/dates/bools via explicit columns=); per-column mask = the parity-hardened Contains predicate (all cuDF workarounds + honest declines inherited). Row op w/ alias-prefix scoping + safelist + builder; polars native (schema dtype gate; lower()-fold literal default — the common viz call never touches regex; rust-guard on regex path; any_horizontal); ComputeMixin twins search_nodes/search_edges (pandas/cuDF; polars frames honest-NIE v1). Matrix: 5-case oracle pins + 4-engine parity-or-NIE + twins test; ledger calls-axis waiver (count_table pattern) + rowops exercised. dgx: 1996 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cypher surface: searchAny(alias, 'term'[, {caseSensitive, regex, columns}]) lifts
at lowering assembly into a search_any pre-filter + fresh marker column (the
pattern-predicate marker mechanism) — composes through AND/OR/NOT; strict opts
validation (unknown keys/bad bools/bad lists/unbound alias -> clear E108);
oracle-pinned cypher-surface cases + 4-engine parity-or-NIE incl. NOT/AND
composition; docs + CHANGELOG.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uDF =~-parity regex guards, null-cell mask, polars E108, $param clear-decline
B1: _lift_search_any_from_row_where now finds calls against a literal/comment-
masked copy of the WHERE text, so searchAny(...) inside a string literal is data
(was: bare .sub rewrote literal contents — silent wrong answer / spurious
unbound-alias error). I5: any residual searchAny( outside literals after the
lift (e.g. $param term) raises a clear E108 instead of leaking downstream.
B2: the cuDF kernel path applies the same decline rules as =~ (_cudf_regex_prep
+ _cudf_casefold_or_decline) before delegating to Contains — NIE on lookaround/
backrefs/inline-flags/unsound folds instead of libcudf crashes or \D->\d
inversion. I1: null cells never match — explicit non-string columns mask
isna() back out after astype(str) ('nan'/'<NA>' artifacts). I2: polars
missing-explicit-column now raises the same E108 as pandas (the comment claimed
a fallback that doesn't exist). Pins for each in the conformance matrix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e + explicit-dtype gate, discriminating float pin, polars E108 pin, docs caveats, rescan-after-skip lift W2-3: polars explicit-column search now canonicalizes Boolean rendering to pandas' 'True'/'False' (cast gave 'true' — a SILENT caseSensitive divergence) with nulls kept null, and declines explicit columns beyond string/int/float/ bool (temporal/categorical/nested stringification is engine-divergent) — NIE, never silent. W2-1: float auto-gate pin made discriminating (7.25 moved to a row with no int '7'). W2-2: polars-engine missing-explicit-column E108 pinned directly. W2-4: docs/CHANGELOG caveat the edge-alias polars NIE + dtype gate. S1: lift rescans from m.start()+1 after an in-literal skip so a quote-spanning pseudo-match can't swallow a real call. S4: two-searchAny composition pin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dgx-measured 86.96% on the CI-verbatim CPU polars lane; floor 79.0 per the baseline's ~8-below convention (also absorbs the wave-2 canonicalize/dtype- gate lines, whose decline/parity branches are pinned in the matrix). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…umns (repr divergence), precedence pin, wording Floats leave the polars _stringify_ok gate: pandas str(1e16)='1e+16' vs the Rust formatter's '1e16' is a silent exponent-regime divergence — the SAME decision the polars toString lowering already made (row_pipeline.py declines Float 'repr diverges'). Pinned NIE for explicit float columns on polars; pandas oracle pin unchanged. Missing-column-beats-dtype-gate precedence pinned (E108, not NIE). Docs/CHANGELOG: float added to the polars decline list; 'folded via lowercase' -> 'case-folded' (pandas upper-folds internally). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… stringify diverges from pandas (dgx-probed)
Probe (rapids 26.02): cudf.Series.astype(str) renders 0.1+0.2 as '0.3' (pandas
'0.30000000000000004'), 1e16 as '1.0e+16' (pandas '1e+16'), and truncates long
mantissas ('123456789.1') — the explicit-columns float path silently diverged
from the pandas oracle. cuDF now declines explicit non-string/int/bool columns
(NIE), symmetric with the polars gate; string/int/bool stay native (bool
parity pinned). Kernel pin added; docs/CHANGELOG updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…date decline (wave-4 S-w4a) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…FrameT/SeriesT (no Any) Use the codebase's cross-engine aliases (compute/typing.py, as predicates/str.py already does): frames are DataFrameT, masks/series are SeriesT; dtype params are "object" (stricter than Any — only passed to the pandas dtype API). Annotation- only; mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…o gate (inspector 2a) Match the streamgl-viz inspector (sortAndFilterRowsByQuery.js shouldSearch): the auto gate only fires on string dtype. A pandas OBJECT column can hold lists/dicts; we included it as 'string' then Contains-over-lists silently never-matched (+ a fillna FutureWarning). Now _has_string_content inspects object columns via infer_dtype and skips non-string content — clean skip, not silent no-match. Real string / StringDtype / cuDF-str columns still searched; polars/cudf already used typed columns (no object ambiguity). Pandas-path fix; pin verifies pandas skip + polars parity. Reference-parity spec: research/searchany-inspector-parity.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
48d529b to
9388be7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
searchAny(entity, term[, opts])— the streamgl-viz inspector's cross-column table search as a composable openCypher-style WHERE predicate, native on all four engines (pandas/cuDF/polars/polars-gpu), plus Python twinsg.search_nodes()/g.search_edges(). Viz-filter L2 (frozen semantics: panel-algebra D2; naming/options via a 4-persona design pass — artifacts underplans/viz-filter-pipeline/research/searchany-persona-pass.md). Stacked on #1679.Semantics (dtype gate is SEMANTICS, not optimization)
regex: trueopt-in obeying the same per-engine decline rules as=~(now enforced on cuDF too: prep + casefold-or-decline before delegating, instead of libcudf crashes or\D→\dinversion)./^[0-9.-]+$/gate); floats/dates/booleans only via the explicitcolumns:list. Explicit non-string columns are served where stringification provably matches the pandas oracle: bool everywhere (polars canonicalizes'True'/'False'), float/date on pandas only — polars and cuDF decline them honestly (dgx-probed: pandasstr(1e16)='1e+16'vs polars'1e16'vs cuDF'1.0e+16', and cuDF truncates mantissas —0.1+0.2 → '0.3').{caseSensitive, regex, columns}strict-validated: unknown keys, non-boolean flags, malformed column lists, and unbound aliases all error clearly; missing explicit columns raise the same E108 on pandas AND polars (pinned to beat the dtype gate in precedence);$paramterms decline with a clear message; null cells never match (incl.astype(str)'s'nan'/'<NA>'artifacts, masked back out).searchAny(...)spelled inside a string literal is data, not a call — the lowering lift matches against a literal/comment-masked copy of the WHERE text (with rescan-after-skip so quote-spanning pseudo-matches can't swallow real calls).searchAnyin one WHERE compose (pinned).Engineering
Per-column matching reuses the parity-hardened
Containspredicate on pandas/cuDF (every cuDF workaround + honest decline from the #1675 review waves inherited) and a fold/any_horizontallowering on polars (Rust-regex guard on the regex path; module-per-familypolars/search.py). Safelisted row op with schema effects; conformance-ledger entries; polars coverage baseline gains asearch.pyfloor (79.0; dgx-measured 86.96 on the CI-verbatim lane). One deliberate base change: the call executor's NIE pass-through extends from polars-only to row-pipeline calls on any engine (is_row_pipeline_call), so kernel-level per-engine declines surface as honestNotImplementedErrorinstead of a wrapped E303 — non-row-pipeline NIEs (e.g.fa2_layoutwithout GPU) still wrap, pinned.Review + verification
5 adversarial findings-mode waves to convergence (waves 4+5 consecutive clean; artifacts
plans/viz-filter-pipeline/waves/pr1680/). Waves fixed 2 BLOCKERs (in-literal lift rewrite; cuDF regex guard bypass) and 5+1 IMPORTANTs (null-cell stringify match, polars E108,$param, polars bool'true'silent divergence, float repr divergence — each now pinned). dgx GB10 (--gpus all, rapids 26.02): targeted 5-file lane 1703 passed at the converged tip; CI green incl. the polars coverage audit. ruff + mypy clean.Boundaries (clear errors, all pinned)
searchAnyin RETURN/WITH projections (WHERE only); edge-aliassearchAny(r, …)declines on polars (pending multi-entity binding-row support); explicit columns beyond string/int/bool decline on polars + cuDF; polars-frame Python twins decline (use the cypher op); regex declines mirror=~.🤖 Generated with Claude Code