Skip to content

acc: make acceptance tests work in Databricks development environments#5662

Open
pietern wants to merge 1 commit into
mainfrom
acc-tests-ignore-local-dev-probes
Open

acc: make acceptance tests work in Databricks development environments#5662
pietern wants to merge 1 commit into
mainfrom
acc-tests-ignore-local-dev-probes

Conversation

@pietern

@pietern pietern commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

The acceptance harness blocks external network access during tests. In some Databricks-internal development environments, local tooling makes network calls that trip the sandbox and fail the suite (CI is unaffected), and a stale local toolchain produced confusing diffs instead of clear errors.

  • Ignore local probes that aren't test traffic (e.g. requests to bare localhost) and disable the local background beacons, so behavior matches CI.
  • Assert the required toolchain up front so a missing/stale tool fails fast: jq, uv, ruff, and python >= 3.11 (provisioned via uv).

Testing: full direct acceptance suite passes locally in a Databricks dev environment; unit tests for the tool checks and python provisioning included.

This pull request and its description were written by Isaac.

@github-actions

Copy link
Copy Markdown
Contributor

Waiting for approval

Based on git history, these people are best suited to review:

  • @denik -- recent work in acceptance/internal/, libs/testserver/, acceptance/

Eligible reviewers: @andrewnester, @anton-107, @renaudhartert-db, @shreyas-goenka, @simonfaltum

Suggestions based on git history. See OWNERS for ownership rules.

@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 12:55 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 12:55 — with GitHub Actions Inactive
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from 149eac3 to 5c5df59 Compare June 19, 2026 13:02
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:02 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:02 — with GitHub Actions Inactive
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from 5c5df59 to dc7f93b Compare June 19, 2026 13:05
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:06 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:06 — with GitHub Actions Inactive
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from dc7f93b to 8f59ba8 Compare June 19, 2026 13:09
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:10 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:10 — with GitHub Actions Inactive
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
version: "0.8.9"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now run uv find python from the Taskfile instead of assuming the right version exists.

This means we need to make sure uv is available before using the Taskfile.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: a9da026

Run: 27829075558

Env 🟨​KNOWN 🔄​flaky ✅​pass 🙈​skip Time
🟨​ aws linux 1 216 99 3:01
🟨​ aws windows 1 4 214 97 9:09
🟨​ aws-ucws linux 1 2 295 18 3:55
🟨​ aws-ucws windows 1 299 16 3:38
🟨​ azure linux 1 216 98 2:57
🟨​ azure windows 1 218 96 3:03
🟨​ azure-ucws linux 1 299 15 3:48
🟨​ azure-ucws windows 1 301 13 3:15
🟨​ gcp linux 1 215 100 3:02
🟨​ gcp windows 1 217 98 2:36
6 interesting tests: 5 flaky, 1 KNOWN
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🔄​ TestFsCpDirToDirFileNotOverwritten ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestFsCpDirToDirFileNotOverwritten/dbfs_to_dbfs ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestFsCpDirToDirFileNotOverwritten/dbfs_to_uc-volumes 🙈​s 🙈​s 🔄​f ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s
🔄​ TestFsCpFileToDir ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestFsCpFileToDir/dbfs_to_dbfs ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
Top 1 slowest tests (at least 2 minutes):
duration env testname
7:07 aws windows TestSecretsPutSecretStringValue

The acceptance harness blocks external network access during tests. In some
Databricks-internal development environments, local tooling issues network
calls that trip the sandbox and fail the suite even though CI passes. Ignore
the local probes that are not test traffic and disable the background beacons
so tests behave the same as on CI.

Also assert the external toolchain (jq, uv, ruff) and provision python via uv
up front, so a missing or stale tool fails fast with a clear message.

Co-authored-by: Isaac
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from 8f59ba8 to a9da026 Compare June 19, 2026 13:37
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:37 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:37 — with GitHub Actions Inactive
@pietern pietern enabled auto-merge June 19, 2026 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants