Ctadel

Deploy the central webhook router

Ctadel runs one stack per customer (e.g. acme.ctadel.eu, bigco.ctadel.eu). But there is only one Ctadel GitHub App — its webhook URL is fixed, so it can't point at N customer subdomains. The webhook router sits in front:

                  ONE GitHub App "Ctadel Twin"
                  webhook → webhook.ctadel.eu
                                │
                                ▼
                  ┌─────────────────────────────────┐
                  │  webhook.ctadel.eu              │
                  │  - verify GitHub HMAC           │
                  │  - lookup installation.id → tenant
                  │  - forward to <tenant>.ctadel.eu│
                  │  - inter-service HMAC           │
                  └────┬───────────┬───────────────┘
                       │           │
                       ▼           ▼
              acme.ctadel.eu  bigco.ctadel.eu …
              /webhook/git    /webhook/git

The router only routes — it never holds the RSA private key, and it's not on the hot path for git clones or PR comments (each customer stack mints its own installation tokens locally).

Step 1, generate the shared secrets

The router and every customer stack share two secrets:

openssl rand -hex 32   # → CTADEL_INSTALL_SIGNING_KEY (signs install-flow state)
openssl rand -hex 32   # → CTADEL_INTER_SVC_SECRET   (signs forwarded webhooks)

Plus the router-only GitHub webhook secret (the one you set when creating the App):

openssl rand -hex 32   # → CTADEL_GH_WEBHOOK_SECRET (router only)

Store all three in your secrets manager — they get distributed to:

SecretRouterEach customer stack
CTADEL_GH_WEBHOOK_SECRET
CTADEL_INSTALL_SIGNING_KEY
CTADEL_INTER_SVC_SECRET

Step 2, deploy the router

The router lives in gateway/ at the repo root with its own docker-compose.gateway.yml. It's independent of any customer stack and binds only to 127.0.0.1 — your existing host Caddy reverse-proxies webhook.ctadel.eu to it.

cd gateway
cp .env.example .env
# fill in CTADEL_GH_APP_SLUG, the three secrets, CTADEL_TENANT_BASE_DOMAIN
docker compose -f docker-compose.gateway.yml up -d --build

Then add the Caddy block. The repo provides one ready to copy in:

cat gateway/Caddyfile.snippet >> /etc/caddy/Caddyfile
systemctl reload caddy

The snippet declares webhook.ctadel.eu and reverse-proxies everything to localhost:9080. Caddy obtains a Let's Encrypt cert automatically on first request.

Confirm it's live:

curl -fsS https://webhook.ctadel.eu/healthz
# → ok

If your Caddy itself runs in a container (not on the host), skip the loopback port mapping and add the Caddy container to the gateway_default Docker network created by the compose. The reverse_proxy upstream then becomes ctadel-webhook-router:8080.

The router persists its installation_id → tenant mapping to a JSON file on a Docker volume (router-data:/data/router.json). Back this up regularly — losing it means every customer has to re-install the App.

Step 3, configure each customer stack

When provisioning a tenant (e.g. acme.ctadel.eu), set in their .env.prod:

CTADEL_TENANT_ID=acme
CTADEL_WEBHOOK_ROUTER_URL=https://webhook.ctadel.eu
CTADEL_INSTALL_SIGNING_KEY=<same value as the router>
CTADEL_INTER_SVC_SECRET=<same value as the router>

The stack's frontend uses CTADEL_WEBHOOK_ROUTER_URL to redirect users to the router when they click Install on GitHub; the stack's orchestrator uses CTADEL_INTER_SVC_SECRET to verify forwarded webhooks.

Stacks do not need CTADEL_GH_WEBHOOK_SECRET.

Step 4, configure the GitHub App's webhook + setup URLs

Both of these point to the router, not to any specific tenant:

App fieldValue
Webhook URLhttps://webhook.ctadel.eu/webhook/git
Setup URLhttps://webhook.ctadel.eu/callback
Webhook secretthe value of CTADEL_GH_WEBHOOK_SECRET on the router

If the App was already created pointing at a specific tenant, edit it on github.com/settings/apps/<slug>.

Step 5, smoke test end-to-end

  1. As an admin on a test customer stack (say acme.ctadel.eu), open Settings → Integrations.
  2. Click Install Ctadel on GitHub. Browser:
    • hits acme.ctadel.eu/api/integrations/github/install,
    • is redirected to webhook.ctadel.eu/install?state=<signed by acme>,
    • is redirected to github.com/apps/ctadel-twin/installations/new?state=<re-signed by router>.
  3. Complete the install on a test GitHub org. GitHub redirects back to webhook.ctadel.eu/callback?installation_id=…&state=….
  4. The router persists the mapping and bounces the browser to acme.ctadel.eu/api/integrations/github/callback?installation_id=….
  5. The Acme stack creates the integration row in its own DB. The Settings page shows "GitHub App installed".
  6. Push a commit to a covered repo. GitHub POSTs webhook.ctadel.eu/webhook/git. Router verifies, forwards to acme.ctadel.eu/webhook/git with X-Ctadel-Forwarded-Sig. Acme's orchestrator verifies the inter-service HMAC and dispatches the scan.

Tail logs on both:

# Router
webhook-router listening on :8080, base_domain=ctadel.eu, db=/data/router.json
install callback: installation=42 → tenant=acme
# Acme stack
Git scan queued: a1b2c3d4 for Xetactf/ctadel-iac-test (main) PR#0 checks=[iac secrets sca]

Inspect the mapping

curl -H "Authorization: Bearer $ROUTER_ADMIN_TOKEN" \
     https://webhook.ctadel.eu/admin/installations | jq .

What happens when a customer uninstalls

GitHub fires installation.deleted to the router. The router removes the mapping row from its JSON store. Subsequent webhooks for that installation_id return ignored / unknown_installation to GitHub.

The customer stack also receives the installation event (forwarded by the router) and can react locally (mark the integration as last_error="uninstalled").

Troubleshooting

  • unknown_installation in router logs after a fresh install — the install flow didn't go through the router (e.g. the customer pasted the GitHub App install URL directly). Always start from the customer stack's Install on GitHub button.
  • invalid state on /install or /callbackCTADEL_INSTALL_SIGNING_KEY differs between the router and the customer stack, or the state expired (>10 min between click and install). Re-sync the env var across all hosts.
  • Forwarded webhooks 401CTADEL_INTER_SVC_SECRET differs between router and the stack. Same fix.
  • Router DB corrupted — restore router-data:/data/router.json from backup. As a last resort, customers can re-install (uninstall → install on GitHub).

Scaling out

For now the router is a single instance with a JSON-file store. Scale path when needed:

  1. Replace the JSON store with Postgres (multi-instance, point-in-time backups).
  2. Put the router behind a load balancer; multiple replicas read the same DB.
  3. Add rate limiting at the LB — GitHub doesn't retry forever but a flood of webhooks shouldn't take everything down.

What's next