[Feature] : API ENDPOINTS PR 5 : Result Data and Baseline Approval#1134
Open
pulk17 wants to merge 6 commits into
Open
[Feature] : API ENDPOINTS PR 5 : Result Data and Baseline Approval#1134pulk17 wants to merge 6 commits into
pulk17 wants to merge 6 commits into
Conversation
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Please prefix your pull request with one of the following: [FEATURE] [FIX] [IMPROVEMENT].
In raising this pull request, I confirm the following (please check boxes):
My familiarity with the project is as follows (check one):
Feature: Results Data and Baseline Approvals (PR 5/6)
Executive Summary
This Pull Request is Part 5 of 6 in the initiative to introduce a fully-featured JSON REST API.
This PR implements the most technically demanding portion of the CI lifecycle: Test Results Analysis and Algorithmic Baseline Management. It allows developers to securely query raw algorithmic outputs (standard out/error), visualize computed diffs between their outputs and the expected baselines, and formally approve new algorithmic outputs as the accepted standard going forward.
Architectural Additions & Enhancements
1. Results Endpoints (
mod_api/routes/results.py)Mounted at
/api/v1/runs, this router exposes granular test execution data:GET /runs/<run_id>/samples/<sample_id>/results: Fetches the hierarchical result structure for a given sample, including whichRegressionTestOutputcomponents failed and why.GET /runs/<run_id>/samples/<sample_id>/results/<result_id>/diff: A specialized endpoint that retrieves the exact textual difference between the CI run's output and the accepted baseline.GET /runs/<run_id>/samples/<sample_id>/results/<result_id>/file: Streams the raw unadulterated output file (e.g., standard error logs, parsed CC data) generated by the algorithm during the run.POST /runs/<run_id>/samples/<sample_id>/baseline-approval: The critical workflow endpoint. Allows users withbaselines:writescope to promote a successful output hash to become the new global baseline for a specific regression test. Includes protections to automatically cascade approvals across test variants if requested.2. Core Diff Service (
mod_api/services/diff_service.py)3. Schemas & Serialization (
mod_api/schemas/results.py)TestResultandTestResultFilemodels.Testing & Quality Assurance
test_services_diff_service.pyto ensure diff generation logic is rock solid regardless of whether files are stored locally or in Google Cloud.isort,pydocstyle,pycodestyle, andmypypass perfectly on all introduced files.Next Steps
Following the review and merge of this PR, the final PR 6 (Swagger UI & Documentation) will be submitted to expose the OpenAPI interface for all these completed endpoints!