Improved the performance of the configuration profiles status summary#48873
Draft
getvictor wants to merge 10 commits into
Draft
Improved the performance of the configuration profiles status summary#48873getvictor wants to merge 10 commits into
getvictor wants to merge 10 commits into
Conversation
Member
Author
|
@coderabbitai full review |
Member
Author
|
/agentic_review |
Contributor
✅ Action performedFull review finished. |
Contributor
There was a problem hiding this comment.
Warning
- Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.
Pull request overview
This PR addresses timeouts in GET /configuration_profiles/summary for Windows at large scale by introducing a per-host rollup table (host_mdm_windows_profiles_status) that materializes each host’s aggregate profiles delivery status, and updating the summary query to read that rollup instead of performing an O(hosts × profiles) correlated aggregation.
Changes:
- Add
host_mdm_windows_profiles_statustable + migration backfill to store one status bucket per Windows host. - Maintain the rollup incrementally across Windows profile write paths, and add an hourly reconcile job as a self-healing sweep.
- Update/extend tests to validate rollup correctness and to reconcile before reading summaries in tests that seed profile rows directly.
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| changes/48340-windows-config-profiles-summary-scale | Release note / changes entry (content excluded by policy). |
| server/service/integration_mdm_test.go | Reconcile rollup before asserting Windows profile summaries in integration tests. |
| server/mock/datastore_mock.go | Add mock hook for ReconcileWindowsProfilesStatus. |
| server/fleet/datastore.go | Extend datastore interface with ReconcileWindowsProfilesStatus. |
| server/datastore/mysql/schema.sql | Add host_mdm_windows_profiles_status table to schema. |
| server/datastore/mysql/migrations/tables/20260707150000_AddHostMDMWindowsProfilesStatus.go | Migration to create/backfill the rollup table. |
| server/datastore/mysql/migrations/tables/20260707150000_AddHostMDMWindowsProfilesStatus_test.go | Migration test for rollup backfill logic. |
| server/datastore/mysql/microsoft_mdm.go | Rollup maintenance helper, reconcile implementation, and summary query changes to read rollup. |
| server/datastore/mysql/microsoft_mdm_test.go | Add/adjust tests to validate rollup maintenance and reconcile behavior. |
| server/datastore/mysql/mdm.go | Ensure Windows MDM cleanup also clears the new rollup table. |
| server/datastore/mysql/mdm_test.go | Reconcile rollup before Windows summary assertions in tests that seed rows directly. |
| server/datastore/mysql/hosts.go | Ensure host deletion cleanup includes host_mdm_windows_profiles_status. |
| cmd/fleet/cron.go | Add hourly cleanup job to reconcile Windows profiles status rollup. |
Files excluded by content exclusion policy (1)
- changes/48340-windows-config-profiles-summary-scale
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (16)
WalkthroughThis PR adds Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ast-grep (0.44.1)server/datastore/mysql/hosts_test.goast-grep timed out on this file server/service/integration_mdm_test.goast-grep timed out on this file Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Code Review by Qodo
1.
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
…s-status-rollup # Conflicts: # server/datastore/mysql/schema.sql
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #48873 +/- ##
==========================================
- Coverage 68.11% 68.11% -0.01%
==========================================
Files 3709 3698 -11
Lines 235142 235288 +146
Branches 12498 12344 -154
==========================================
+ Hits 160168 160264 +96
- Misses 60598 60620 +22
- Partials 14376 14404 +28
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…s-status-rollup # Conflicts: # server/datastore/mysql/schema.sql
Member
Author
|
@coderabbitai full review |
Member
Author
|
/agentic_review |
Contributor
✅ Action performedFull review finished. |
Contributor
There was a problem hiding this comment.
Warning
- Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.
Pull request overview
Copilot reviewed 15 out of 16 changed files in this pull request and generated 4 comments.
Files excluded by content exclusion policy (1)
- changes/48340-windows-config-profiles-summary-scale
Comment on lines
+1870
to
+1874
| // windowsHostProfileStatusCaseExpr returns the SQL CASE expression (and its as-yet-unexpanded args) | ||
| // that reduces a group of host_mdm_windows_profiles rows (aliased hmwp) to a single status bucket: | ||
| // `<statusPrefix>failed`, `<statusPrefix>pending`, `<statusPrefix>verifying`, `<statusPrefix>verified`, | ||
| // or ”. It is the single source of truth for the Windows profile status priority logic (failed > | ||
| // pending > verifying > verified, reserved profiles excluded, install-only for verifying/verified, |
Comment on lines
+1956
to
+1959
| // orphan a rollup row) safe without wrapping it in a transaction. | ||
| // - The orphan-delete removes the now-stale rollup row for a host whose last profile row was just | ||
| // deleted (the upsert's GROUP BY produces no row for it), which the summary treats as ”. | ||
| // |
Comment on lines
+12
to
+14
| // Up_20260707150000 creates host_mdm_windows_profiles_status, a per-host rollup of the aggregate Windows configuration-profile | ||
| // delivery status. It materializes exactly one bucket per host ('failed'|'pending'|'verifying'|'verified'|”) | ||
| func Up_20260707150000(tx *sql.Tx) error { |
|
Code review by qodo was updated up to the latest commit e899bbb |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue: Resolves #48340
Checklist for submitter
If some of the following don't apply, delete the relevant line.
Changes file added for user-visible changes in
changes/,orbit/changes/oree/fleetd-chrome/changes.See Changes files for more information.
Input data is properly validated,
SELECT *is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters.Timeouts are implemented and retries are limited to avoid infinite loops
If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes
Testing
Added/updated automated tests
Where appropriate, automated tests simulate multiple hosts and test for host isolation (updates to one hosts's records do not affect another)
QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
Database migrations
COLLATE utf8mb4_unicode_ci).New Fleet configuration settings
If you didn't check the box above, follow this checklist for GitOps-enabled settings:
fleetctl generate-gitopsfleetd/orbit/Fleet Desktop
runtime.GOOSis used as needed to isolate changesSummary by CodeRabbit
New Features
Bug Fixes