diff --git a/README.md b/README.md index caa0ec4..a1ee0dc 100644 --- a/README.md +++ b/README.md @@ -1,39 +1,184 @@ # SDLC Orchestrator -Lightweight webhook-driven orchestrator that replaces n8n for the Gitea + Coder + Claude Code SDLC pipeline. - -Receives Gitea webhooks, routes them to the correct SDLC stage, creates Coder workspaces, and posts status comments back on issues/PRs. +Lightweight webhook-driven orchestrator for the Gitea + Coder + Claude Code SDLC pipeline. Receives Gitea webhooks, creates ephemeral Coder workspaces running Claude Code, and cleans up when done. ## Architecture ``` -Gitea Webhooks → SDLC Orchestrator → Coder API (creates workspace) - → Gitea API (posts comments) +Gitea Webhook → Orchestrator → Coder API (create workspace) + → Gitea API (post status comment) + +Workspace runs Claude Code → reads issue → does work → posts results to Gitea + → POST /webhook/task-complete/:name (callback) + +Orchestrator receives callback → deletes workspace via Coder API ``` -~500 lines of TypeScript. No database, no UI, no state — just a webhook router. +## Adding a New Project -## Webhook Endpoints +To add a new Gitea repo to the SDLC pipeline: -| Endpoint | Gitea Event | Action | -|----------|-------------|--------| -| `POST /webhook/gitea-issue-triage` | Issue opened | Route to `analyse` or `fix-bug` | -| `POST /webhook/gitea-issue-label` | Issue labeled | Stage transition (architect/develop/test/devops) | -| `POST /webhook/gitea-issue-comment` | Comment created | Re-trigger analyst if still in analysis | -| `POST /webhook/gitea-pr-review` | PR opened/synced | Trigger code review | -| `POST /webhook/gitea-pr-review-rework` | Review submitted | Route to `rework-pr` or `test` | -| `POST /webhook/gitea-release` | Release issue | Tag RC or production release | -| `GET /health` | — | Health check | +### 1. Add Claude Code slash commands to the repo -Plus a cron job (Monday 9 AM) for weekly maintenance. +Create `.claude/commands/` in the repo with command files for each SDLC stage: -## Setup +``` +.claude/commands/ +├── analyse.md # Business analyst — requirements & acceptance criteria +├── architect.md # Systems architect — design doc & spec +├── develop.md # Developer — implement & create PR +├── review-code.md # Reviewer — approve or request changes +├── test.md # QA — test report & additional tests +├── fix-bug.md # Bug fix — skip analyse/architect +├── rework-pr.md # Address review feedback +├── release.md # Tag RC or production release +├── devops.md # Migration & deployment +└── maintenance.md # Weekly maintenance tasks +``` + +Each file uses this format: + +```markdown +--- +description: What this command does +allowed-tools: Read, Bash, Glob, Grep +--- + +You are a **Role Name**. Your job is to... + +## Process + +1. **Read the issue**: + ```bash + curl -s "https://gitea.samson.media/api/v1/repos/${GITEA_ORG}/${GITEA_REPO}/issues/$ARGUMENTS" \ + -H "Authorization: token ${GITEA_TOKEN}" | jq '{title, body, labels: [.labels[].name]}' + ``` + +2. **Do your work...** + +3. **Post a comment**: + ```bash + curl -X POST "https://gitea.samson.media/api/v1/repos/${GITEA_ORG}/${GITEA_REPO}/issues/$ARGUMENTS/comments" \ + -H "Authorization: token ${GITEA_TOKEN}" \ + -H "Content-Type: application/json" \ + -d '{"body": "..."}' + ``` +``` + +`$ARGUMENTS` is replaced with the issue/PR number at runtime. + +### 2. Add bot users as collaborators + +Add both `claude-dev` and `claude-review` as **direct collaborators** with **Write** access on the repo. Org membership alone is not sufficient. + +### 3. Configure Gitea webhooks + +In the repo's **Settings → Webhooks**, create these webhooks: + +| # | URL | Events | +|---|-----|--------| +| 1 | `https://sdlc.samson.media/webhook/gitea-issue-triage` | Issues: `opened` | +| 2 | `https://sdlc.samson.media/webhook/gitea-issue-label` | Issues: `labeled` | +| 3 | `https://sdlc.samson.media/webhook/gitea-issue-comment` | Issue Comment: `created`, `edited` | +| 4 | `https://sdlc.samson.media/webhook/gitea-pr-review` | Pull Request: `opened`, `synchronized` | +| 5 | `https://sdlc.samson.media/webhook/gitea-pr-review-rework` | Pull Request Review: `submitted` | +| 6 | `https://sdlc.samson.media/webhook/gitea-release` | Issues: `opened`, `labeled` | + +All webhooks use: +- Content type: `application/json` +- Method: `POST` + +### 4. (Optional) Add to maintenance cron + +To include the repo in weekly maintenance, add it to the `MAINTENANCE_REPOS` env var: + +``` +MAINTENANCE_REPOS=org1/repo1=https://gitea.samson.media/org1/repo1.git,org2/repo2=https://gitea.samson.media/org2/repo2.git +``` + +### 5. Test it + +Create an issue in the repo. The orchestrator will: +1. Receive the `issues/opened` webhook +2. Create a Coder workspace running `/analyse` +3. Claude reads the issue, posts clarifying questions as a comment +4. Workspace is deleted automatically when done + +## SDLC Flow + +| Stage | Trigger | Task Type | Bot Account | +|-------|---------|-----------|-------------| +| Analyse | Issue opened / comment | `analyse` | claude-dev | +| Architect | Label: `ready-for-architecture` | `architect` | claude-dev | +| Develop | Label: `ready-for-development` | `develop` | claude-dev | +| Review | PR opened/updated | `review-code` | claude-review | +| Rework | Review: changes requested | `rework-pr` | claude-dev | +| Test | Review: approved | `test` | claude-review | +| Deploy | Label: `ready-for-deployment` | `devops` | claude-dev | +| Fix Bug | Issue opened with `bug` label | `fix-bug` | claude-dev | + +## Infrastructure Setup ### Prerequisites -- Node.js 22+ -- Coder running with the `cloudflare-worker` template (see `coder/`) -- Gitea with two bot accounts: `claude-dev` and `claude-review` +- k3s cluster with Traefik ingress and cert-manager +- Coder instance (e.g. `coder.samson.media`) +- Gitea instance (e.g. `gitea.samson.media`) +- Docker registry (e.g. `registry.samson.media`) +- Claude Code Max subscription (for OAuth token) + +### Deploy the Orchestrator + +1. **Build the workspace image** (pre-installed Node.js, Claude Code, Playwright deps): + ```bash + docker build -f coder/workspace.Dockerfile -t registry.samson.media/coder-workspace:latest . + docker push registry.samson.media/coder-workspace:latest + ``` + +2. **Push the Coder template**: + ```bash + coder templates push cloudflare-worker --directory coder/cloudflare-worker --yes + ``` + Note the template ID from the Coder dashboard. + +3. **Create Gitea bot accounts**: + - `claude-dev` — used for most stages (analyse, develop, etc.) + - `claude-review` — used for review and test stages + - Generate API tokens for each + +4. **Generate Claude Code OAuth token** (uses Max subscription instead of API credits): + ```bash + claude setup-token + ``` + This opens a browser for auth and outputs a long-lived token (`sk-ant-oat01-...`). + +5. **Create k8s secrets**: + ```bash + kubectl create namespace sdlc-orchestrator + kubectl create secret generic orchestrator-secrets -n sdlc-orchestrator \ + --from-literal=CODER_URL=https://coder.samson.media \ + --from-literal=CODER_TOKEN= \ + --from-literal=CODER_TEMPLATE_ID= \ + --from-literal=GITEA_DEV_TOKEN= \ + --from-literal=GITEA_REVIEW_TOKEN= \ + --from-literal=CLAUDE_OAUTH_TOKEN= \ + --from-literal=CALLBACK_URL=https://sdlc.samson.media \ + --from-literal=GITEA_URL=https://gitea.samson.media \ + --from-literal=BOT_DEV_USERNAME=claude-dev \ + --from-literal=BOT_REVIEW_USERNAME=claude-review + ``` + +6. **Deploy**: + ```bash + # Tag to trigger CI build + git tag -a v1.0.0 -m "Initial release" + git push origin main --tags + + # Or deploy manually + docker build -t registry.samson.media/sdlc-orchestrator:latest . + docker push registry.samson.media/sdlc-orchestrator:latest + kubectl apply -f k8s/deployment.yaml + ``` ### Environment Variables @@ -43,58 +188,58 @@ Plus a cron job (Monday 9 AM) for weekly maintenance. | `CODER_URL` | Yes | Coder API base URL | | `CODER_TOKEN` | Yes | Coder session token | | `CODER_TEMPLATE_ID` | Yes | Coder workspace template ID | -| `GITEA_DEV_TOKEN` | Yes | Gitea API token for `claude-dev` | -| `GITEA_REVIEW_TOKEN` | Yes | Gitea API token for `claude-review` | -| `ANTHROPIC_API_KEY` | Yes | Anthropic API key for Claude Code | +| `GITEA_DEV_TOKEN` | Yes | Gitea API token for claude-dev | +| `GITEA_REVIEW_TOKEN` | Yes | Gitea API token for claude-review | +| `CLAUDE_OAUTH_TOKEN` | Yes | Claude Code Max OAuth token | +| `CALLBACK_URL` | Yes | Public URL of this service (e.g. `https://sdlc.samson.media`) | | `GITEA_URL` | No | Gitea base URL (default: `https://gitea.samson.media`) | | `BOT_DEV_USERNAME` | No | Dev bot username (default: `claude-dev`) | | `BOT_REVIEW_USERNAME` | No | Review bot username (default: `claude-review`) | +| `QUEUE_CONCURRENCY` | No | Max concurrent workspaces (default: 2) | | `MAINTENANCE_REPOS` | No | Comma-separated `org/repo=clone_url` for weekly maintenance | -### Local Development +> **Important**: Do NOT set `ANTHROPIC_API_KEY` in the workspace — it takes priority over the OAuth token and will use pay-per-use credits instead of your Max subscription. -```bash -npm install -cp k8s/secret.yaml.example .env # Edit with real values (use KEY=value format) -npm run dev -``` +## Webhook Endpoints -### Deploy to k3s +| Endpoint | Purpose | +|----------|---------| +| `GET /health` | Health check | +| `GET /queue` | Queue status (pending, running, active workspaces) | +| `POST /webhook/gitea-issue-triage` | Route new issues to analyse or fix-bug | +| `POST /webhook/gitea-issue-label` | Stage transitions via labels | +| `POST /webhook/gitea-issue-comment` | Re-trigger analyst on new/edited comments | +| `POST /webhook/gitea-pr-review` | Trigger code review on PR open/update | +| `POST /webhook/gitea-pr-review-rework` | Route review outcomes to rework or test | +| `POST /webhook/gitea-release` | Handle release workflow | +| `POST /webhook/task-complete/:name` | Callback from workspaces when done | -```bash -# Build and push image -docker build -t registry.samson.media/sdlc-orchestrator:latest . -docker push registry.samson.media/sdlc-orchestrator:latest +## How Workspaces Work -# Create secrets -cp k8s/secret.yaml.example k8s/secret.yaml -# Edit k8s/secret.yaml with real values -kubectl apply -f k8s/secret.yaml +Workspaces are **ephemeral** (emptyDir, no persistent storage): -# Deploy -kubectl apply -f k8s/deployment.yaml -``` +1. Orchestrator creates workspace via Coder API with parameters (issue number, task type, tokens, callback URL) +2. Startup script clones the repo and checks out `develop` +3. For heavy stages (develop, test, review, rework, fix-bug, devops): installs deps, builds frontend, installs Playwright +4. For lightweight stages (analyse, architect, release, maintenance): skips build +5. Reads `.claude/commands/.md`, strips YAML frontmatter, substitutes `$ARGUMENTS` with issue number +6. Runs `claude -p --dangerously-skip-permissions --verbose ""` +7. On completion (success or failure via ERR/EXIT trap): POSTs to callback URL +8. Orchestrator receives callback, deletes workspace via Coder API -### Configure Gitea Webhooks +## Resilience -For each project, add these webhooks in **Settings → Webhooks**: +- **Deduplication**: Tasks keyed by `{taskType}-{repo}-{issue}`. Duplicate webhooks are dropped. +- **Startup reconciliation**: On boot, queries Coder for running workspaces and re-adopts them into the queue. +- **409 handling**: If workspace already exists — adopts if running, deletes and retries if stopped/failed. +- **Callback trap**: Startup script uses `trap` to always fire the callback, even on clone failure or other errors. +- **TTL safety net**: Coder template has 1-hour auto-stop as a backstop for missed callbacks. -| Event | URL | -|-------|-----| -| Issues (opened) | `https://sdlc.samson.media/webhook/gitea-issue-triage` | -| Issues (labeled) | `https://sdlc.samson.media/webhook/gitea-issue-label` | -| Issue Comments (created) | `https://sdlc.samson.media/webhook/gitea-issue-comment` | -| Pull Request (opened, synchronized) | `https://sdlc.samson.media/webhook/gitea-pr-review` | -| Pull Request Review (submitted) | `https://sdlc.samson.media/webhook/gitea-pr-review-rework` | -| Issues (opened, labeled) — releases | `https://sdlc.samson.media/webhook/gitea-release` | +## CI/CD -## Coder Template +| Workflow | Trigger | Output | +|----------|---------|--------| +| `publish.yml` | `v*` tags | `registry.samson.media/sdlc-orchestrator:` + `:latest` | +| `publish-workspace.yml` | Changes to `coder/workspace.Dockerfile` on main | `registry.samson.media/coder-workspace:latest` | -The `coder/` directory contains the Terraform template for ephemeral Kubernetes workspaces. See `coder/README.md`. - -## Migrating from n8n - -1. Deploy this service to k3s -2. Update Gitea webhooks to point to `sdlc.samson.media` instead of `n8n.samson.media` -3. Verify with a test issue -4. Decommission n8n +Fleet GitOps (in `samson-media/devops` repo) watches `registry.samson.media/sdlc-orchestrator:latest` for deployment.