add tool to gitea

2026-06-05 02:34:00 +10:00
commit 532b912ffb
25 changed files with 4981 additions and 0 deletions
@@ -0,0 +1,566 @@
+# kforge
+
+**kforge** eliminates Kubernetes boilerplate. You define your app once in a `kforge.yml` file — environments, infrastructure, ingress, TLS, DNS — and kforge generates production-ready flat manifests on every CI run. Nothing is committed to your repo except the config.
+
+Built for self-hosted MicroK8s, Gitea Actions, Cloudflare DNS, cert-manager, and CNPG — but designed to be extended.
+
+---
+
+## How it works
+
+1. Add a `kforge.yml` to your repo root describing your app, environments, and infrastructure.
+2. kforge generates Kubernetes YAML at CI time — Service, Deployment, Ingress, cert-manager Certificates, CronJobs, and infrastructure (database, cache, storage, queue, search).
+3. Generated manifests are applied to the cluster and discarded. Only `kforge.yml` is committed.
+
+```
+kforge.yml  →  kforge generate  →  kubectl apply  →  cluster
+```
+
+---
+
+## Installation
+
+### Option A — Build from source (recommended for self-hosted runners)
+
+```bash
+git clone https://gitea.yourdomain.com/yourorg/kforge.git
+cd kforge
+go build -o /usr/local/bin/kforge .
+```
+
+Install once on your Gitea runner host and it's available to every repo automatically.
+
+### Option B — Gitea Package Registry
+
+If you've published kforge to your Gitea instance's generic package registry:
+
+```bash
+curl -fsSL \
+  -H "Authorization: token $GITEA_TOKEN" \
+  "https://gitea.yourdomain.com/api/packages/yourorg/generic/kforge/latest/kforge-linux-amd64" \
+  -o /usr/local/bin/kforge
+chmod +x /usr/local/bin/kforge
+```
+
+---
+
+## Quick start
+
+**1. Add `kforge.yml` to your repo:**
+
+```yaml
+meta:
+  name: my-app
+  tenant: my-org
+
+registry:
+  url: registry.yourdomain.com
+
+dns:
+  provider: cloudflare
+  cloudflare:
+    api_token: ${CLOUDFLARE_API_TOKEN}
+    zones:
+      - name: yourdomain.com
+        zone_id: ${CF_ZONE_ID_YOURDOMAIN}
+  node_ip: ${KFORGE_NODE_IP}
+
+cluster:
+  tls_issuer: letsencrypt-prod
+  cnpg:
+    host: cnpg-main-rw.default.svc.cluster.local
+
+defaults:
+  port: 3000
+  health_check:
+    path: /healthcheck
+
+infrastructure:
+  database:
+    provider: cnpg
+  cache:
+    provider: valkey
+    mode: standalone
+
+environments:
+  staging:
+    namespace: staging
+    image_tag: latest
+    env_vars:
+      - name: API_URL
+        value: https://api-staging.yourdomain.com
+        type: plain
+    ingress:
+      hosts:
+        - hostname: app-staging.yourdomain.com
+          tls: true
+          dns_record: true
+      auth:
+        enabled: true
+        users:
+          - yourname
+
+  production:
+    namespace: production
+    image_tag: latest
+    env_vars:
+      - name: API_URL
+        value: https://api.yourdomain.com
+        type: plain
+    ingress:
+      hosts:
+        - hostname: app.yourdomain.com
+          tls: true
+          dns_record: true
+    infrastructure:
+      cache:
+        mode: cluster
+        replicas: 3
+```
+
+**2. Validate your config and see what secrets you need:**
+
+```bash
+kforge validate
+kforge secrets list
+```
+
+**3. Preview what gets generated:**
+
+```bash
+kforge generate --env production --dry-run
+```
+
+**4. Generate your Gitea Actions workflow:**
+
+```bash
+kforge gitea-actions > .gitea/workflows/deploy.yml
+```
+
+---
+
+## CLI reference
+
+### `kforge validate`
+
+Parses `kforge.yml`, checks structural correctness, and verifies all required secrets are present in the current environment. Exits non-zero if anything is wrong — use this as the first step in CI to fail fast before touching the cluster.
+
+```bash
+kforge validate
+kforge validate -c path/to/kforge.yml
+```
+
+---
+
+### `kforge generate`
+
+Generates flat Kubernetes manifests for one or all environments. Writes files to `.kforge-out/` by default, or prints to stdout with `--dry-run`.
+
+```bash
+kforge generate                          # all environments
+kforge generate --env staging            # one environment
+kforge generate --env production --dry-run
+kforge generate --env production --output .kube/
+```
+
+**Output files per environment:**
+
+| File                              | Contents                                             |
+| --------------------------------- | ---------------------------------------------------- |
+| `{env}-core.yaml`                 | Service, Deployment, Ingress, Certificates, CronJobs |
+| `{env}-infra-cnpg-database.yaml`  | CNPG Database CR                                     |
+| `{env}-infra-cnpg-role.yaml`      | CNPG DatabaseRole CR                                 |
+| `{env}-infra-cache.yaml`          | Valkey/Redis Deployment or StatefulSet               |
+| `{env}-infra-cache-svc.yaml`      | Cache Service                                        |
+| `{env}-infra-storage.yaml`        | Minio Deployment                                     |
+| `{env}-infra-queue-nats.yaml`     | NATS Deployment                                      |
+| `{env}-infra-search.yaml`         | Meilisearch Deployment                               |
+| `{env}-infra-servicemonitor.yaml` | Prometheus ServiceMonitor CR                         |
+
+Infrastructure env vars (`DATABASE_URL`, `CACHE_URL`, `STORAGE_ENDPOINT`, etc.) are automatically injected into the Deployment — you don't wire these up manually.
+
+---
+
+### `kforge secrets list`
+
+Prints the full secrets checklist for this repo, grouped by where each secret needs to live, with live status checks against the current environment.
+
+```bash
+kforge secrets list
+```
+
+Example output:
+
+```
+── Gitea org secret ──
+  DOCKER_USERNAME                          ✗ missing
+  CLOUDFLARE_API_TOKEN                     ✓ set
+  CF_ZONE_ID_YOURDOMAIN_COM                ✓ set
+  KFORGE_NODE_IP                           ✓ set
+  SOPS_AGE_KEY                             ✗ missing
+
+── Gitea repo secret ──
+  KUBE_HOST                                ✓ set
+  KUBE_TOKEN                               ✓ set
+  KUBE_CERTIFICATE                         ✓ set
+
+── Cluster secret (auto-generated) ──
+  prod-my-org-my-app-db-credentials        — managed by kforge
+  prod-my-org-my-app-cache-credentials     — managed by kforge
+```
+
+---
+
+### `kforge secrets apply`
+
+Generates secure random credentials and creates Kubernetes Secrets in the cluster for all enabled infrastructure services. Safe to run on every deploy — secrets are only created if they don't already exist. Use `--force` to rotate credentials.
+
+```bash
+kforge secrets apply --env staging
+kforge secrets apply --env production --force   # rotates all credentials
+```
+
+For basic auth secrets, kforge prints the generated passwords once at apply time. Save them — they are not stored anywhere else.
+
+Requires `KUBE_HOST`, `KUBE_TOKEN`, and `KUBE_CERTIFICATE` to be set (Gitea injects these automatically during CI).
+
+---
+
+### `kforge dns ensure`
+
+Creates or updates Cloudflare DNS A records for all ingress hosts with `dns_record: true`. Idempotent — no-ops if the record already points to the correct IP.
+
+```bash
+kforge dns ensure --env staging
+kforge dns ensure --env staging --env production
+```
+
+Requires `CLOUDFLARE_API_TOKEN` and `KFORGE_NODE_IP` to be set.
+
+---
+
+### `kforge gitea-actions`
+
+Generates a complete `.gitea/workflows/deploy.yml` for this app. Re-run whenever you add environments or change deploy configuration.
+
+```bash
+kforge gitea-actions
+kforge gitea-actions --output .gitea/workflows/deploy.yml
+kforge gitea-actions --branch main --env staging --env production
+```
+
+The generated workflow runs these steps for each environment, in order:
+
+1. Build and push Docker image
+2. `kforge validate`
+3. `kforge secrets apply` — creates missing cluster secrets
+4. `kforge dns ensure` — creates missing DNS records
+5. `kforge generate` — writes manifests to `.kforge-out/`
+6. `kubectl apply` — applies core manifests
+7. `kubectl apply` — applies infra manifests
+8. `kubectl rollout restart` — triggers rolling update
+
+---
+
+## `kforge.yml` reference
+
+### Naming and interpolation
+
+kforge generates resource names using the pattern `{env_prefix}-{tenant}-{name}` (e.g. `prod-my-org-my-app`). Use `${tokens}` anywhere in string values to reference resolved fields:
+
+| Token           | Resolves to                                 |
+| --------------- | ------------------------------------------- |
+| `${name}`       | `meta.name`                                 |
+| `${tenant}`     | `meta.tenant`                               |
+| `${env}`        | current environment key                     |
+| `${env_prefix}` | short env prefix (first 4 chars, or custom) |
+| `${full_name}`  | `{env_prefix}-{tenant}-{name}`              |
+| `${namespace}`  | resolved namespace for the environment      |
+
+Environment variables (e.g. `${CLOUDFLARE_API_TOKEN}`) are resolved from the CI process environment at generation time — never hardcode secrets in `kforge.yml`.
+
+---
+
+### `meta`
+
+```yaml
+meta:
+  name: my-app # required — short app name, lowercase, hyphens ok
+  tenant: my-org # required — org/tenant identifier
+  name_override: ~ # optional — override the full generated resource name
+  previous_name: ~ # optional — set when renaming; kforge patches rather than recreates
+```
+
+---
+
+### `registry`
+
+```yaml
+registry:
+  url: registry.yourdomain.com # default: registry.natelubitz.com
+  repository: my-org/my-app # default: {tenant}/{name}
+  pull_secret: regcred # default: regcred
+```
+
+Override per-environment by adding a `registry:` block under the environment.
+
+---
+
+### `dns`
+
+```yaml
+dns:
+  provider: cloudflare # currently supported: cloudflare
+  cloudflare:
+    api_token: ${CLOUDFLARE_API_TOKEN}
+    zones:
+      - name: yourdomain.com
+        zone_id: ${CF_ZONE_ID_YOURDOMAIN}
+    proxied: false # false = DNS-only, required for cert-manager DNS-01
+  node_ip: ${KFORGE_NODE_IP} # IP for new A records
+  skip_dns: false # true to disable all DNS management
+```
+
+kforge matches each ingress hostname to the correct zone by longest-suffix match — add one zone entry per domain you own.
+
+---
+
+### `cluster`
+
+```yaml
+cluster:
+  tls_issuer: letsencrypt-prod # default: letsencrypt-prod
+  ingress_class: nginx # default: nginx
+  cnpg:
+    host: cnpg-main-rw.default.svc.cluster.local # your CNPG cluster service
+  namespace_pattern: "${env}" # default: environment key
+```
+
+---
+
+### `defaults`
+
+All fields can be overridden per-environment.
+
+```yaml
+defaults:
+  port: 3000
+  replicas: 1
+  image_pull_policy: Always
+  service_type: ClusterIP
+  dockerfile: Dockerfile
+  health_check:
+    path: /healthcheck
+    port: 3000
+    initial_delay_seconds: 15
+    period_seconds: 10
+    timeout_seconds: 5
+    failure_threshold: 3
+    liveness: true
+    readiness: true
+  resources:
+    requests:
+      cpu: "100m"
+      memory: "128Mi"
+    limits:
+      cpu: "500m"
+      memory: "512Mi"
+  env_vars: []
+```
+
+---
+
+### `infrastructure` (root defaults)
+
+Define infrastructure at the root level and it applies to **all environments** by default. Per-environment blocks are a shallow merge — only the fields you specify override; everything else inherits.
+
+```yaml
+infrastructure:
+  database:
+    provider: cnpg # only supported provider currently
+  cache:
+    provider: valkey # valkey (recommended) | redis
+    mode: standalone # standalone | cluster
+    replicas: 1
+  storage:
+    enabled: false
+    provider: minio # standalone | distributed
+  queue:
+    enabled: false
+    provider: nats # nats (recommended, ~20MB) | rabbitmq (~200MB)
+  search:
+    enabled: false
+    provider: meilisearch
+  monitoring:
+    enabled: false
+    provider: prometheus
+    metrics_path: /metrics
+```
+
+If a service block exists at root, it is **enabled by default** for all environments. To disable it for a specific environment:
+
+```yaml
+environments:
+  dev:
+    infrastructure:
+      cache:
+        enabled: false
+```
+
+To scale up for production while staging uses the root defaults:
+
+```yaml
+environments:
+  production:
+    infrastructure:
+      cache:
+        mode: cluster
+        replicas: 3 # provider: valkey inherited from root
+```
+
+**Injected env vars per service** (automatically added to your Deployment):
+
+| Service          | Env vars injected                                                         |
+| ---------------- | ------------------------------------------------------------------------- |
+| database (CNPG)  | `DATABASE_URL`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` |
+| cache            | `CACHE_URL`, `CACHE_HOST`, `CACHE_PORT`, `CACHE_PASSWORD`                 |
+| storage          | `STORAGE_ENDPOINT`, `STORAGE_ACCESS_KEY`, `STORAGE_SECRET_KEY`            |
+| queue (NATS)     | `QUEUE_URL`, `QUEUE_HOST`                                                 |
+| queue (RabbitMQ) | `QUEUE_URL`, `QUEUE_USER`, `QUEUE_PASSWORD`                               |
+| search           | `SEARCH_URL`, `SEARCH_MASTER_KEY`                                         |
+
+---
+
+### `environments`
+
+```yaml
+environments:
+  production:
+    namespace: production
+    replicas: 1
+    image_tag: latest # override with --set image_tag=$SHA in CI
+    env_prefix: prod # default: first 4 chars of env key
+
+    env_vars:
+      - name: API_URL
+        type: plain
+        value: https://api.yourdomain.com
+
+      - name: SOME_SECRET
+        type: secret_ref # pull from an existing Kubernetes Secret
+        secret_name: my-secrets
+        secret_key: some_secret
+
+      - name: FEATURE_FLAG
+        type: configmap_ref # pull from a ConfigMap
+        configmap_name: my-config
+        configmap_key: feature_flag
+
+    ingress:
+      hosts:
+        - hostname: app.yourdomain.com
+          tls: true # kforge generates a cert-manager Certificate
+          dns_record: true # kforge creates a Cloudflare A record
+      auth:
+        enabled: false # enable for staging/dev to protect unreleased work
+        users:
+          - yourname # passwords are auto-generated by kforge secrets apply
+
+    infrastructure:
+      # shallow merge on top of root — only override what differs
+      cache:
+        mode: cluster
+        replicas: 3
+
+    cron_jobs:
+      - name: cleanup
+        schedule: "0 2 * * *"
+        command: ["node", "scripts/cleanup.js"]
+        inherit_env: true # inherits all deployment env vars
+        env_vars:
+          - name: BATCH_SIZE
+            value: "500"
+            type: plain
+        resources:
+          requests:
+            cpu: "50m"
+            memory: "64Mi"
+          limits:
+            cpu: "200m"
+            memory: "256Mi"
+        restart_policy: OnFailure
+        concurrency_policy: Forbid
+
+    lifecycle:
+      delete: false # if true + previous_name set, deletes old resources
+      delete_grace_seconds: 300 # 5-minute countdown before deletion runs in CI
+```
+
+---
+
+## Secrets architecture
+
+kforge works with three categories of secrets, each living in the right place for its scope.
+
+### Category A — Gitea org secrets
+
+Set once at the organisation level; available to every repo automatically.
+
+| Secret                 | Purpose                                        |
+| ---------------------- | ---------------------------------------------- |
+| `DOCKER_USERNAME`      | Registry authentication                        |
+| `DOCKER_PASSWORD`      | Registry authentication                        |
+| `CLOUDFLARE_API_TOKEN` | DNS record management (Zone:Read + DNS:Edit)   |
+| `CF_ZONE_ID_{DOMAIN}`  | One per zone, e.g. `CF_ZONE_ID_YOURDOMAIN_COM` |
+| `KFORGE_NODE_IP`       | MicroK8s node IP for DNS A records             |
+| `SOPS_AGE_KEY`         | Decrypts `.kforge/secrets.enc.yml`             |
+
+### Category B — Gitea repo secrets
+
+Per-repo, since different apps may deploy to different clusters.
+
+| Secret             | Purpose                       |
+| ------------------ | ----------------------------- |
+| `KUBE_HOST`        | Kubernetes API server URL     |
+| `KUBE_TOKEN`       | Service account token         |
+| `KUBE_CERTIFICATE` | Base64-encoded CA certificate |
+
+### Category C — Cluster secrets (auto-generated)
+
+Created by `kforge secrets apply`. Never appear in Gitea or in `kforge.yml`.
+
+| Secret name                       | Contents                          |
+| --------------------------------- | --------------------------------- |
+| `{full_name}-db-credentials`      | CNPG-managed database credentials |
+| `{full_name}-basic-auth`          | htpasswd for ingress basic auth   |
+| `{full_name}-cache-credentials`   | Valkey/Redis password             |
+| `{full_name}-storage-credentials` | Minio access/secret keys          |
+| `{full_name}-queue-credentials`   | RabbitMQ credentials              |
+| `{full_name}-search-credentials`  | Meilisearch master key            |
+
+Run `kforge secrets list` at any time to see the full checklist with live status for your current repo.
+
+---
+
+## Project structure
+
+```
+kforge.yml                        ← your app config (committed)
+.gitea/
+  workflows/
+    deploy.yml                    ← generated by kforge gitea-actions (committed)
+.kforge/
+  secrets.enc.yml                 ← SOPS-encrypted sensitive config (committed)
+  kforge.age                      ← age private key (NEVER committed — goes in Gitea as SOPS_AGE_KEY)
+```
+
+Generated manifests (`.kforge-out/`) are never committed — they are created at CI time and discarded after `kubectl apply`.
+
+---
+
+## Planned
+
+- `kforge serve` — web UI for managing infrastructure across all repos
+- `kforge lifecycle rename` — interactive rename flow with resource patching
+- Route53 and Porkbun DNS providers
+- `--set key=value` flag for runtime overrides (e.g. `--set image_tag=$SHA`)