Skip to content

feat: add OSS OpenSearch and semantic FTS support#815

Open
jamesgao-jpg wants to merge 18 commits into
zilliztech:mainfrom
jamesgao-jpg:fts_v2_backend
Open

feat: add OSS OpenSearch and semantic FTS support#815
jamesgao-jpg wants to merge 18 commits into
zilliztech:mainfrom
jamesgao-jpg:fts_v2_backend

Conversation

@jamesgao-jpg

Copy link
Copy Markdown
Collaborator

Summary

This PR adds the backend and frontend support needed for unfiltered full-text search benchmarking while deferring publication of filtered FTS results.

PR #794 introduced the preceding full-text search benchmark foundation used by this work. This PR supersedes #814 with a reduced scope that excludes filtered FTS frontend and result support.

Changes

  • add the OSS OpenSearch backend client, configuration, CLI registration, and focused tests
  • retain unfiltered semantic FTS metrics and controls
  • retain the standard FTS QPS, recall, NDCG, MRR, and load frontend views
  • remove filtered FTS datasets, cases, runners, backend paths, CLI options, tests, result artifacts, and frontend views
  • preserve current upstream backend and compatibility changes by incorporating the latest main

Deferred Scope

Filtered FTS search results and their frontend presentation are intentionally deferred and are not published by this PR.

Verification

  • make lint
  • make unittest
  • 32 focused FTS/OpenSearch tests
  • DCO verification for every commit in upstream/main..fts_v2_backend
  • VDBBench pre-push gate and impact-map validation
  • confirmed that no filtered FTS symbols or filtered result artifacts remain in the outgoing diff

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Remove filtered FTS dataset, runner, backend, frontend, test, and published
result changes while retaining unfiltered OSS OpenSearch and semantic FTS
support. Remove the task-local session progress file from the branch.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Preserve current upstream backends and compatibility changes while retaining
the unfiltered FTS and OSS OpenSearch work from fts_ver2.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jamesgao-jpg
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jamesgao-jpg

This comment was marked as outdated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants