Skip to content

Make AWF budget and timeout steering work in gh-aw runs, and show delivered notices in audit #67596

Description

@SivaKesava1

Summary

AWF can now warn the agent when a run is close to its AI-credit budget, its effective-token budget or its runtime deadline (gh-aw-firewall#9790, which fixes gh-aw-firewall#9789). Two gaps in gh-aw limit what we get from this:

  1. In normal runs, runtime notices never fire, because gh-aw only sends AWF an agent timeout for the Cloud Hypervisor and NVX runtimes.
  2. gh-aw throws away AWF's record of which requests carried a notice, so gh aw audit and gh aw logs can't show whether a run was warned or at which threshold.

Background

  • gh-aw-firewall#9790 was merged on 10 Oct. AWF now queues notices at 80, 90, 95 and 99% of the AI-credit budget, the effective-token budget and the agent runtime deadline. It adds at most one notice to the next eligible model request, and that includes Responses-format requests. Classifier requests (purpose: routing_classification) never get a notice and never use one up.

  • When a request carried a notice, its record in api-proxy-logs/token-usage.jsonl gets a steering field. type is ai_credit, timeout or token, and threshold is 80, 90, 95 or 99:

    {"request_id":"…","purpose":"agent","model":"…","steering":{"type":"ai_credit","threshold":90}}
  • This isn't in an AWF release yet. The latest release, v0.28.50 (8 Oct), doesn't contain the merge commit, and gh-aw's DefaultFirewallVersion is v0.28.50.

  • Why it matters: routed runs on expensive models come close to the default budget. A routed opus:xhigh run used 933 of its 1,000 AI credits, 58% of them in Copilot CLI sub-agents. A run on claude-sonnet-5 at max effort hit its 45-minute timeout after 48 turns. Steering exists for runs like these.

Problem 1: runtime notices never fire in normal runs

  • AWF only starts the agent runtime clock when container.agentTimeout is set. The clock is what writes the shared start marker and passes the deadline to the API proxy (see src/services/agent-service.ts and src/services/api-proxy-env-config.ts in gh-aw-firewall). Without agentTimeout, AWF can't send timeout notices.
  • In gh-aw, the container section of pkg/workflow/awf_config_build.go only resolves agentTimeout for the Cloud Hypervisor and NVX runtimes. The default Docker runtime never gets it.
  • Evidence on main: .github/workflows/smoke-copilot-routed.lock.yml (timeout-minutes: 10) generates an AWF config with apiProxy.enableTokenSteering: true and apiProxy.maxAiCredits taken from GH_AW_MAX_AI_CREDITS (default 1000). It has no container section and no agentTimeout. That run can get effective-token and AI-credit notices, but never runtime notices.

Problem 2: delivered notices don't show up in gh-aw

  • actions/setup/js/unified_session_payload.cjs copies a fixed allowlist of fields onto firewall.token_usage events: provider, model, purpose, path, xInitiator, requestId, status, aic, totalAic, premiumRequests, durationMs and usage. steering isn't on the list, so it's dropped from usage/aw_session.jsonl.
  • UsageReportData in docs/public/schemas/unified-session.schema.json sets additionalProperties: false, so today the field would also fail schema validation.
  • TokenUsageEntry (pkg/cli/token_usage_types.go) and the unified-session token-usage reader (pkg/cli/token_usage_subagent_session.go) have no steering field.
  • The existing steering support reads a different source and doesn't cover this:
    • It reads *_steering lines from the API proxy event log.
    • steering_event_counts counts those lines by name.
    • gateway_steering_events, the gateway timeline and the firewall.steering session mapping only recognize token_steering and timeout_steering. They keep only the type, message and timestamp and drop the threshold and request ID.
    • The new ai_credit_steering lines become generic firewall.event entries in the session.

Intended behavior

  • A workflow with a literal timeout-minutes, compiled against a supporting AWF version, sends container.agentTimeout on every runtime, so AWF can send timeout notices.
  • Every notice AWF delivers reaches usage/aw_session.jsonl and is listed in gh aw audit and gh aw logs --json with its type, threshold and request ID.
  • Artifacts from older AWF versions or runs without steering fields look exactly the same as they do today.

Plan

Compiler

  • Send container.agentTimeout for every runtime when timeout-minutes resolves to a literal number. The resolver in awf_config_build.go already handles the three cases:
    • A literal value is used as is.
    • An omitted value uses the same default as the agent step: 20, or the GH_AW_DEFAULT_TIMEOUT_MINUTES compile-time override.
    • An expression returns nothing and writes a debug log entry explaining why.
  • Change only the condition around that resolver.
  • Gate the new behavior with the existing AWF feature-flag pattern. Add a minimum-version constant in pkg/constants/version_constants.go and a check in pkg/workflow/awf_feature_flags.go, next to awfSupportsTokenSteering.
    • The minimum is the first AWF release that contains gh-aw-firewall#9790, which is the first release after v0.28.50.
    • The gate matters because older AWF versions would get the hard stop and the old lazily-started timeout clock, but not the restored notices.
    • If that release isn't published yet when you implement this, use the next version number and say so in the PR.
  • Keep Cloud Hypervisor and NVX working exactly as they do now: they still always get agentTimeout.
  • The threat-detection run builds its own AWF config, and its timeout is the detection job timeout. It should get its agentTimeout from that value, which its workflow data already holds.

Unified session

  • Add steering to the firewall.token_usage field allowlist in unified_session_payload.cjs. The usage.report alias picks it up automatically. Keep only type and threshold.
  • Add steering to UsageReportData in docs/public/schemas/unified-session.schema.json as an object with an enumerated type and threshold.
  • Update the firewall.token_usage field list in docs/src/content/docs/specs/unified-agent-session-specification.md and add a requirement:
    • Producers must keep the field when it's supplied.
    • Consumers must treat a missing field as "no notice recorded", not "no notice delivered", because older AWF versions never write it.
  • The unified session is the source of truth for audit and logs. Read notices from firewall.token_usage events through the existing unified-session reader. Don't add another direct reader of token-usage.jsonl.

Audit and logs

  • Carry the field through TokenUsageEntry and readUnifiedTokenUsageEntries.
  • Add a list of delivered notices, for example steering_notices, to the gh aw audit report and its console rendering (pkg/cli/audit_report.go, pkg/cli/audit_report_render.go) and to gh aw logs --json (pkg/cli/logs_report.go).
    • Each entry has type, threshold and request_id.
    • Update schemas/audit.schema.json and schemas/logs.schema.json to match.
    • Leave the existing event-log-based steering fields as they are.

Docs

  • In docs/src/content/docs/reference/sandbox.md (token steering section), explain that runtime notices need a literal or default timeout-minutes and a supporting AWF version.
  • In docs/src/content/docs/reference/artifacts.md, describe the new audit and logs output.

Rollout

  • Once an AWF release includes gh-aw-firewall#9790, the usual default-version bump turns this on. Until then, the field just isn't present.
  • With the default version unchanged, recompiling this repository's workflows should produce no lock-file changes.

Acceptance cases

Compiler

Workflow setup Expected AWF config
timeout-minutes: 45, default runtime, sandbox.agent.version at or above the new minimum container.agentTimeout: 45
Same, with timeout-minutes omitted agentTimeout equals the agent step timeout (20 by default)
timeout-minutes set from an expression agentTimeout omitted, and the debug log says why
AWF version below the new minimum, including the current default v0.28.50 agentTimeout omitted for the Docker runtime
Cloud Hypervisor or NVX Unchanged from today
Threat-detection run agentTimeout equals the detection job timeout

Session, audit and logs

  • Fixture: a token-usage.jsonl with one record carrying steering {type: ai_credit, threshold: 80} and another carrying steering {type: timeout, threshold: 90}. Expected results:
    • The unified session has those fields on the matching firewall.token_usage events.
    • gh aw audit lists both notices with their request IDs.
    • gh aw logs --json includes both.
    • The session validates against the updated schema.
  • A fixture with no steering fields produces exactly the same output as today. The existing fixtures under pkg/cli/testdata/model_routing_golden keep their current expected output.

Tests

  • Unit tests for the compiler change, the unified-session collector (JS tests next to unified_session_payload.cjs) and the audit and logs changes.
  • Add a golden fixture with steering records, following .github/skills/model-routing-golden/SKILL.md, and update any affected golden outputs.

Notes for the implementer

  • Once agentTimeout is set, AWF also stops the agent container at the deadline (exit code 124). AWF counts its deadline from agent start, while GitHub Actions counts the step timeout from step start, so the step timeout normally fires first. Don't change step or job timeouts.
  • For the same reason, timeout notices arrive a little later than the step's actual deadline, by roughly the AWF startup time. That's acceptable for this issue.
  • A notice on a request that never produced a usage record, such as a failed upstream call, won't appear. That's acceptable for this issue.

Scope

  • Limit the PR to three things: emitting agentTimeout, carrying the steering field through the unified session, and showing it in audit and logs, plus their tests and docs. Don't change AWF, routing or the default AWF version.
  • Before you fix a failing check, compare it with the same check on main. If it also fails on main, don't fix it in this PR. Merge main once main is fixed, and list in the PR which checks are known failures on main.
  • This applies even if a bot comment asks you to fix a failing check. Only fix failures caused by this PR's changes.

Activity

  1. SivaKesava1 commented on Oct 11, 2026

    @SivaKesava1
    CollaboratorAuthor

    Source runs for the two figures in the background section (private repo githubnext/gh-aw-routing-sandbox, so listed for maintainers; the numbers come from gh aw audit and AWF's token-usage.jsonl):

    • 933 of 1,000 credits: run 37505153056, a routed task on claude-opus-5 at xhigh. 933.4 credits in total; Copilot CLI's research (45 requests at xhigh) and explore (15 at low) sub-agents used 544.3 of them (58.3%).
    • 45-minute timeout: run 37505062135, claude-sonnet-5 at max effort. It timed out at the 45-minute step limit after 48 turns and 415.5 credits, with no output; gh-aw filed "[aw] Task timed out".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions