Autonomous Incident Response · Google Antigravity SDK · Cloud Run

Event-Driven SRE Root Cause Analysis Agent

An autonomous Site Reliability Engineering (SRE) Root Cause Analysis agent running on Google Cloud Run, built with the Google Antigravity Python SDK (google-antigravity) and operational skill packs from github.com/google/skills.

google-antigravity Python SDK
Cloud Monitoring → Pub/Sub → Cloud Run
Firestore Alert-Storm Deduplication
Read-Only Sandbox & Policy Hooks

Architecture

From Cloud Monitoring Alert to Verified Root Cause Report

When a Cloud Monitoring alert fires, an OIDC-authenticated Pub/Sub push subscription wakes the scale-to-zero Cloud Run service. After acquiring an idempotent Firestore lock for the incident ID, incident_commander orchestrates three domain-scoped subagents in an isolated per-incident workspace and emits a schema-validated RCAReport.

Alert-Storm Deduplication

IncidentStateStore runs a Firestore transaction on sre_incidents/{incident_id} so cascading alert notifications or Pub/Sub retries never trigger duplicate LLM investigations.

Bundled `google/skills` Packs

Subagents mount curated skill directories (skills/telemetry/ and skills/infra/) via SubagentCapabilities(skills_paths=...), combining structured Python tools with CLI playbooks.

Confidence-Gated Publishing

RCAPublisher always logs the structured RCAReport to Cloud Logging, and only comments on the suspected GitHub Pull Request or opens an incident issue when confidence_score >= 0.75.

Security & Guardrails

Defense-in-Depth Against Prompt Injection in Logs & Commits

An SRE agent inspects untrusted external text (application logs, HTTP paths, commit messages). Rather than relying on system-prompt instructions alone, src/tinker_sre/safety.py enforces a strict read-only security posture through the Antigravity SDK's policy and hook engine:

Default-Deny Policy Chain

Starts from policy.deny_all() and policy.workspace_only(), explicitly blocks create_file and edit_file, and allowlists only read-only built-ins (view_file, list_directory, search_directory, find_file) and the 7 read-only diagnostic tools.

Token-Validated Shell Predicate

policy.allow("run_command", when=is_readonly_diagnostic_command) runs inside RunCommandConfig(enable_sandbox=True): it rejects subshells ($(...), backticks), unquoted pipes/redirections, and mutating gcloud/git verbs, failing closed on any parse error.

Structured Output

Schema-Enforced `RCAReport`

LocalAgentConfig(response_schema=RCAReport) guarantees every investigation finishes with a machine-actionable Pydantic report:

{
  "incident_id": "inc-2026-1108-8841",
  "service_name": "checkout-api",
  "root_cause_category": "CODE_REGRESSION",
  "confidence_score": 0.94,
  "executive_summary": "5xx latency and ConnectionPoolExhausted errors began 90s after revision checkout-api-00042 deployed commit 9f3c1a2 (PR #184), which removed the httpx AsyncClient timeout.",
  "timeline": [
    {"timestamp_utc": "2026-11-08T18:12:04Z", "source": "GIT", "description": "PR #184 merged to main (commit 9f3c1a2)"},
    {"timestamp_utc": "2026-11-08T18:14:19Z", "source": "DEPLOYMENT", "description": "Cloud Run revision checkout-api-00042 shifted to 100% traffic"},
    {"timestamp_utc": "2026-11-08T18:15:48Z", "source": "LOGS", "description": "Error Reporting spike: PoolTimeout in src/payments/client.py:74"}
  ],
  "remediation_steps": [
    {
      "priority": "IMMEDIATE_MITIGATION",
      "description": "Roll back Cloud Run traffic to healthy revision checkout-api-00041.",
      "suggested_command": "gcloud run services update-traffic checkout-api --region=us-central1 --to-revisions=checkout-api-00041=100"
    }
  ]
}