Skip to content
This repository was archived by the owner on Jul 3, 2026. It is now read-only.

Tighten Codex service failure classification#59

Merged
Pigbibi merged 2 commits into
mainfrom
codex/tighten-service-failure-classification
Jul 3, 2026
Merged

Tighten Codex service failure classification#59
Pigbibi merged 2 commits into
mainfrom
codex/tighten-service-failure-classification

Conversation

@Pigbibi

@Pigbibi Pigbibi commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • tighten auth/config failure matching to explicit phrases
  • avoid treating source-code words like token/secret/allow as auth failures
  • add regression coverage for source snippet classification

Tests

  • python3 -m ruff check scripts/run_monthly_codex_audit.py scripts/codex_audit_service.py service/ai_gateway_service.py tests/test_run_monthly_codex_audit.py
  • python3 -m unittest tests.test_run_monthly_codex_audit.RunMonthlyCodexAuditTests.test_service_failure_classification_identifies_infra_failures tests.test_run_monthly_codex_audit.RunMonthlyCodexAuditTests.test_service_failure_classification_ignores_source_code_secret_words
  • python3 -m unittest discover -s tests
  • git diff --check

Co-Authored-By: Codex <noreply@openai.com>
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

🤖 Codex PR Review

⚠️ Review skipped: The Codex review could not be completed.

Codex service request failed: 401 {"error": "OIDC workflow_ref is not allowed"}

Please ensure a human reviewer checks this PR before merging.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 32d159e504

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +675 to +689
auth_config_signals = (
"permission denied",
"unauthorized",
"forbidden",
"oidc",
"missing bearer",
"missing token",
"invalid token",
"bad credentials",
"not allowed",
"allowlist",
"api key is required",
"no api key configured",
"secret is missing",
"secret not configured",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore auth classification for clone authentication failures

When the service clone step fails with GitHub’s usual fatal: authentication failed ... message, _prepare_repo() wraps it as git clone failed ..., but this new allowlist no longer matches that phrase. Those jobs are now marked unknown_failure, so run_monthly_codex_audit.py will skip the infra-failure path and, for auto, can fall back to API remediation instead of reporting the service auth/config outage. Please include the explicit authentication-failure wording here and in the mirrored classifiers.

Useful? React with 👍 / 👎.

Comment on lines +184 to +188
"allowlist",
"api key is required",
"no api key configured",
"secret is missing",
"secret not configured",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify request-time service config failures

For request-time 401s there is no async job failure_category, so run_codex_service() relies on this tuple to decide whether to return the infra exit code. The service still emits config errors such as CODEX_AUDIT_SERVICE_ALLOWED_REPOSITORIES is required before a job is created; after replacing the broad allow match with only allowlist, those messages become unknown_failure, and auto runs can proceed to API fallback instead of posting the service-infra failure comment. Add explicit signals for these required service allowlist/config errors.

Useful? React with 👍 / 👎.

@Pigbibi Pigbibi merged commit ee3047c into main Jul 3, 2026
4 checks passed
@Pigbibi Pigbibi deleted the codex/tighten-service-failure-classification branch July 3, 2026 08:17

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 428d439280

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"oidc",
"missing bearer",
"missing token",
"invalid token",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify expired service tokens as auth failures

When the script-based audit service rejects a request because the GitHub OIDC JWT is past exp, it raises PermissionError("Token expired") (scripts/codex_audit_service.py:557), and the client wraps that 401 body before this classifier runs. Because the narrowed signals only include missing token/invalid token, that request-time auth outage is now unknown_failure, so auto runs can fall back to API remediation instead of posting the service-infrastructure failure. Please keep an explicit token expired signal here and in the mirrored service classifier.

Useful? React with 👍 / 👎.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant