infra/kforge
2
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases 1 Wiki Activity
Files
renovate/golang-1.x
kforge/readme.md
T
nate.lubitz 565a91e235
Publish Action Image / build (push) Successful in 33s
Details
Publish Action Image / publish (push) Successful in 28s
Details
update validation
2026-06-29 16:58:52 +10:00

22 KiB
Raw Permalink Blame History

kforge

kforge eliminates Kubernetes boilerplate. You define your app once in a kforge.yml file — environments, infrastructure, ingress, TLS, DNS — and kforge generates production-ready flat manifests on every CI run. Nothing is committed to your repo except the config and generated workflows.

Built for self-hosted MicroK8s, Gitea Actions, external-dns, cert-manager, and CNPG — but designed to be extended.

How it works

  1. Add a kforge.yml to your repo root describing your app, environments, and infrastructure.
  2. kforge generates Kubernetes YAML at CI time — Service, Deployment, Ingress, cert-manager Certificates, CronJobs, and infrastructure (database, cache, storage, queue, search).
  3. Generated manifests are applied to the cluster and discarded. Only kforge.yml and generated workflow files are committed.
kforge.yml  →  kforge generate  →  kubectl apply  →  cluster

DNS records are managed by external-dns running in the cluster — kforge writes the appropriate annotations on the Ingress and external-dns creates the records automatically. TLS is handled by cert-manager reading those same Certificate CRs.

Installation

Option A — Build from source (recommended for self-hosted runners)

git clone https://gitea.yourdomain.com/yourorg/kforge.git
cd kforge
go build -o /usr/local/bin/kforge .

Install once on your Gitea runner host and it's available to every repo automatically.

Option B — Gitea Package Registry

If you've published kforge to your Gitea instance's generic package registry:

curl -fsSL \
  -H "Authorization: token $GITEA_TOKEN" \
  "https://gitea.yourdomain.com/api/packages/yourorg/generic/kforge/latest/kforge-linux-amd64" \
  -o /usr/local/bin/kforge
chmod +x /usr/local/bin/kforge

Quick start

1. Add kforge.yml to your repo:

meta:
  name: my-app
  tenant: my-org

registry:
  url: registry.yourdomain.com

dns:
  target: ${KFORGE_NODE_IP}   # your cluster node's public IP

cluster:
  tls_issuer: letsencrypt-prod
  cnpg:
    host: cnpg-main-rw.default.svc.cluster.local

defaults:
  port: 3000
  health_check:
    path: /healthcheck

infrastructure:
  database:
    provider: cnpg
  cache:
    provider: valkey
    mode: standalone

environments:
  staging:
    namespace: staging
    image_tag: latest
    env_vars:
      - name: API_URL
        value: https://api-staging.yourdomain.com
        type: plain
    ingress:
      hosts:
        - hostname: app-staging.yourdomain.com
          tls: true
          dns_record: true   # external-dns creates this record
      auth:
        enabled: true
        users:
          - yourname

  production:
    namespace: production
    image_tag: latest
    env_vars:
      - name: API_URL
        value: https://api.yourdomain.com
        type: plain
    ingress:
      hosts:
        - hostname: app.yourdomain.com
          tls: true
          dns_record: true
    infrastructure:
      cache:
        mode: cluster
        replicas: 3

2. Validate your config and see what secrets you need:

kforge validate
kforge secrets list

3. Preview what gets generated:

kforge generate --env production --dry-run

4. Generate your Gitea Actions workflows:

kforge gitea-actions > .gitea/workflows/deploy.yml
kforge gitea-preview > .gitea/workflows/preview.yml   # optional — PR preview environments

Adding kforge to an existing project

Step 1 — Add kforge.yml and generate workflows

Add kforge.yml to your repo root (see Quick start above), then:

kforge gitea-actions       # → .gitea/workflows/deploy.yml
kforge gitea-preview       # → .gitea/workflows/preview.yml  (if preview.enabled: true)

Commit kforge.yml and both generated workflow files.

Step 2 — Gitea secrets

Org-level (set once, shared across all repos):

Secret Value
DOCKER_USERNAME Registry login username
DOCKER_PASSWORD Registry login password/token
KFORGE_NODE_IP Your cluster's public node IP (used as the external-dns record target)

Repo-level (per project):

Secret How to get it
KUBE_HOST kubectl config view --raw -o jsonpath='{.clusters[0].cluster.server}'
KUBE_CERTIFICATE kubectl config view --raw -o jsonpath='{.clusters[0].cluster.certificate-authority-data}'
KUBE_TOKEN See below

Create a long-lived deploy service account:

kubectl create serviceaccount kforge-deployer -n kube-system
kubectl create clusterrolebinding kforge-deployer \
  --clusterrole=cluster-admin \
  --serviceaccount=kube-system:kforge-deployer
kubectl create token kforge-deployer -n kube-system --duration=8760h

Step 3 — Cluster prerequisites

These must already be running on your cluster:

Component Purpose
NGINX ingress controller Handles ingress_class: nginx
cert-manager + ClusterIssuer named letsencrypt-prod Issues TLS certificates
external-dns Reads Ingress annotations, creates DNS records
CNPG operator Required if infrastructure.database is enabled
regcred imagePullSecret Must exist in each environment namespace

For external-dns, configure it with your DNS provider and --source=ingress. kforge writes the target annotation automatically from dns.target.

Create regcred in each namespace:

kubectl create secret docker-registry regcred \
  --docker-server=registry.yourdomain.com \
  --docker-username=<user> \
  --docker-password=<pass> \
  -n production

Step 4 — First deploy

The workflow is fully automated after setup, but the very first time:

  1. Create the namespace: kubectl create namespace production
  2. Create regcred in that namespace (above).
  3. If using CNPG database, make the CNPG superuser Secret available in the namespace. The default secret name is cnpg-main-superuser — copy it from wherever your CNPG cluster lives: 
    kubectl get secret cnpg-main-superuser -n cnpg-system -o yaml \
  | sed 's/namespace: cnpg-system/namespace: production/' \
  | kubectl apply -f -
  4. Push to main. The workflow runs kforge secrets apply (creates credentials), kforge generate (writes manifests), and kubectl apply.

CLI reference

kforge validate

Parses kforge.yml, checks structural correctness, and verifies all required secrets are present in the current environment. Use this as the first step in CI to fail fast before touching the cluster.

kforge validate
kforge validate -c path/to/kforge.yml

kforge generate

Generates flat Kubernetes manifests for one or all environments. Writes files to .kforge-out/ by default, or prints to stdout with --dry-run.

kforge generate                          # all environments
kforge generate --env staging            # one environment
kforge generate --env production --dry-run
kforge generate --env production --output .kube/
kforge generate --pr-number 42           # preview environment for PR #42

Output files per environment:

File Contents
{env}-core.yaml Service, Deployment, Ingress, Certificates, CronJobs
{env}-infra-cnpg-database.yaml CNPG Database CR
{env}-infra-cnpg-db-init.yaml Job that creates the PostgreSQL role and syncs password
{env}-infra-cache.yaml Valkey/Redis Deployment or StatefulSet
{env}-infra-cache-svc.yaml Cache Service
{env}-infra-storage.yaml Minio Deployment
{env}-infra-queue-nats.yaml NATS Deployment
{env}-infra-search.yaml Meilisearch Deployment
{env}-infra-servicemonitor.yaml Prometheus ServiceMonitor CR

When --pr-number is given, the output also includes a Namespace manifest so kubectl apply is self-contained — no separate namespace creation step needed.

Infrastructure env vars (DATABASE_URL, CACHE_URL, STORAGE_ENDPOINT, etc.) are automatically injected into the Deployment — you don't wire these up manually.

kforge secrets list

Prints the full secrets checklist for this repo, grouped by where each secret needs to live, with live status checks against the current environment.

kforge secrets list

Example output:

── Gitea org secret ──
  DOCKER_USERNAME          ✗ missing
  KFORGE_NODE_IP           ✓ set

── Gitea repo secret ──
  KUBE_HOST                ✓ set
  KUBE_TOKEN               ✓ set
  KUBE_CERTIFICATE         ✓ set

── Cluster secret (auto-generated) ──
  prod-my-org-my-app-db-credentials      — managed by kforge
  prod-my-org-my-app-cache-credentials   — managed by kforge

kforge secrets apply

Generates secure random credentials and creates Kubernetes Secrets in the cluster for all enabled infrastructure services. Safe to run on every deploy — secrets are only created if they don't already exist. Use --force to rotate credentials.

kforge secrets apply --env staging
kforge secrets apply --env production --force   # rotates all credentials
kforge secrets apply --pr-number 42             # apply secrets for PR preview #42

What gets created:

Secret Contents
{full_name}-db-credentials PostgreSQL username (derived from app name) + random alphanumeric password
{full_name}-basic-auth htpasswd entries for ingress basic auth
{full_name}-cache-credentials Valkey/Redis password
{full_name}-storage-credentials Minio access key + secret key
{full_name}-queue-credentials RabbitMQ username + password (NATS needs no credentials)
{full_name}-search-credentials Meilisearch master key

For basic auth secrets, kforge prints the generated passwords once at apply time — save them.

Requires KUBE_HOST, KUBE_TOKEN, and KUBE_CERTIFICATE to be set (Gitea injects these automatically during CI).

kforge gitea-actions

Generates a complete .gitea/workflows/deploy.yml for this app. Re-run whenever you add environments or change deploy configuration.

kforge gitea-actions
kforge gitea-actions --output .gitea/workflows/deploy.yml
kforge gitea-actions --branch main --env staging --env production

The generated workflow runs these steps for each environment, in order:

  1. Build and push Docker image (tagged with git SHA)
  2. kforge validate
  3. kforge secrets apply — creates any missing cluster secrets
  4. kforge generate — writes manifests to .kforge-out/
  5. kubectl apply — applies core manifests (Service, Deployment, Ingress, Certs)
  6. kubectl apply — applies infrastructure manifests (database, cache, etc.)
  7. kubectl rollout restart — triggers rolling update

DNS records are created automatically by external-dns when the Ingress is applied — no separate DNS step needed.

kforge gitea-preview

Generates .gitea/workflows/preview.yml that deploys an ephemeral environment for each pull request.

kforge gitea-preview
kforge gitea-preview --output .gitea/workflows/preview.yml

Requires preview.enabled: true in kforge.yml. See the preview environments section below.

The generated workflow:

  • On PR open / sync: Builds a :pr-{N} tagged image, applies secrets to a new namespace (preview-pr-{N}), generates manifests (including a Namespace resource), and deploys.
  • On PR close: Deletes the preview-pr-{N} namespace, removing all preview resources automatically.

kforge.yml reference

Naming and interpolation

kforge generates resource names using the pattern {env_prefix}-{tenant}-{name} (e.g. prod-my-org-my-app). Use ${tokens} anywhere in string values to reference resolved fields:

Token Resolves to
${name} meta.name
${tenant} meta.tenant
${env} current environment key
${env_prefix} short env prefix (first 4 chars, or custom)
${full_name} {env_prefix}-{tenant}-{name}
${namespace} resolved namespace for the environment

Environment variables (e.g. ${KFORGE_NODE_IP}) are resolved from the CI process environment at generation time — never hardcode secrets in kforge.yml.

meta

meta:
  name: my-app        # required — short app name, lowercase, hyphens ok
  tenant: my-org      # required — org/tenant identifier
  name_override: ~    # optional — override the full generated resource name
  previous_name: ~    # optional — set when renaming; kforge patches rather than recreates

registry

registry:
  url: registry.yourdomain.com   # default: registry.natelubitz.com
  repository: my-org/my-app      # default: {tenant}/{name}
  pull_secret: regcred           # default: regcred

Override per-environment by adding a registry: block under the environment.

dns

DNS records are managed by external-dns inside the cluster. kforge writes annotations on the Ingress resource for each host with dns_record: true:

external-dns.alpha.kubernetes.io/hostname: "app.yourdomain.com"
external-dns.alpha.kubernetes.io/target:   "<dns.target>"

dns:
  target: ${KFORGE_NODE_IP}   # value for the external-dns target annotation
  skip_dns: false             # true = don't write external-dns annotations

target supports token interpolation — ${KFORGE_NODE_IP} is the most common value, resolved from the KFORGE_NODE_IP Gitea org secret at generate time.

cluster

cluster:
  tls_issuer: letsencrypt-prod                        # default: letsencrypt-prod
  ingress_class: nginx                                # default: nginx
  cnpg:
    host: cnpg-main-rw.default.svc.cluster.local     # your CNPG cluster's read-write service
    cluster_name: cnpg-main                           # default: cnpg-main
    superuser_secret: cnpg-main-superuser             # default: cnpg-main-superuser
  namespace_pattern: "${env}"                         # default: environment key

defaults

All fields can be overridden per-environment.

defaults:
  port: 3000
  replicas: 1
  image_pull_policy: Always
  service_type: ClusterIP
  dockerfile: Dockerfile
  health_check:
    path: /healthcheck
    port: 3000
    initial_delay_seconds: 15
    period_seconds: 10
    timeout_seconds: 5
    failure_threshold: 3
    liveness: true
    readiness: true
  resources:
    requests:
      cpu: "100m"
      memory: "128Mi"
    limits:
      cpu: "500m"
      memory: "512Mi"
  env_vars: []

infrastructure (root defaults)

Define infrastructure at the root level and it applies to all environments by default. Per-environment blocks are a shallow merge — only the fields you specify override; everything else inherits.

infrastructure:
  database:
    provider: cnpg   # only supported provider currently
  cache:
    provider: valkey # valkey (recommended) | redis
    mode: standalone # standalone | cluster
    replicas: 1
  storage:
    enabled: false
    provider: minio  # standalone | distributed
  queue:
    enabled: false
    provider: nats   # nats (recommended, ~20MB) | rabbitmq (~200MB)
  search:
    enabled: false
    provider: meilisearch
  monitoring:
    enabled: false
    provider: prometheus
    metrics_path: /metrics

If a service block exists at root, it is enabled by default for all environments. To disable it for a specific environment:

environments:
  dev:
    infrastructure:
      cache:
        enabled: false

To scale up for production while staging uses the root defaults:

environments:
  production:
    infrastructure:
      cache:
        mode: cluster
        replicas: 3   # provider: valkey inherited from root

Injected env vars per service (automatically added to your Deployment):

Service Env vars injected
database (CNPG) DATABASE_URL, DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD
cache CACHE_URL, CACHE_HOST, CACHE_PORT, CACHE_PASSWORD
storage STORAGE_ENDPOINT, STORAGE_ACCESS_KEY, STORAGE_SECRET_KEY
queue (NATS) QUEUE_URL, QUEUE_HOST
queue (RabbitMQ) QUEUE_URL, QUEUE_USER, QUEUE_PASSWORD
search SEARCH_URL, SEARCH_MASTER_KEY

CNPG database details

For a centralized CNPG cluster, kforge generates two resources:

  1. A Database CR — declaratively manages the database lifecycle (CNPG v1.22+).
  2. A db-init Job — runs on every deploy to create the PostgreSQL role (if it doesn't exist) and sync its password from the {full_name}-db-credentials Secret.

kforge secrets apply must run before the Job so the Secret exists. The PostgreSQL username is derived from the app's full name (prod_my_org_my_app). The password is alphanumeric only, which keeps the init Job shell script simple and safe.

The cnpg-main-superuser Secret (created by CNPG for the cluster) must be present in the target namespace. Copy it once during cluster bootstrap or namespace creation.

preview

Enables PR preview environments. Run kforge gitea-preview to generate the workflow.

preview:
  enabled: true
  base_environment: staging          # inherit infra and env vars from this env
  namespace_prefix: preview-pr       # namespace = preview-pr-{PR_NUMBER}
  hostname_template: "pr-${PR_NUMBER}.${name}.yourdomain.com"

Supported tokens in hostname_template: ${PR_NUMBER}, ${name}, ${tenant}.

On PR open/sync, kforge deploys the app to a preview-pr-{N} namespace with:

  • Image tagged :pr-{N} (built from the PR branch)
  • Hostname from the template
  • Infrastructure and env vars inherited from base_environment

On PR close, the entire namespace is deleted.

environments

environments:
  production:
    namespace: production
    replicas: 1
    image_tag: latest           # override with --set image_tag=$SHA in CI
    env_prefix: prod            # default: first 4 chars of env key

    env_vars:
      - name: API_URL
        type: plain
        value: https://api.yourdomain.com

      - name: SOME_SECRET
        type: secret_ref        # pull from an existing Kubernetes Secret
        secret_name: my-secrets
        secret_key: some_secret

      - name: FEATURE_FLAG
        type: configmap_ref     # pull from a ConfigMap
        configmap_name: my-config
        configmap_key: feature_flag

    ingress:
      hosts:
        - hostname: app.yourdomain.com
          tls: true             # kforge generates a cert-manager Certificate
          dns_record: true      # external-dns creates this record
      auth:
        enabled: false          # enable for staging/dev to protect unreleased work
        users:
          - yourname            # passwords are auto-generated by kforge secrets apply

    infrastructure:
      # shallow merge on top of root — only override what differs
      cache:
        mode: cluster
        replicas: 3

    cron_jobs:
      - name: cleanup
        schedule: "0 2 * * *"
        command: ["node", "scripts/cleanup.js"]
        inherit_env: true       # inherits all deployment env vars
        env_vars:
          - name: BATCH_SIZE
            value: "500"
            type: plain
        resources:
          requests:
            cpu: "50m"
            memory: "64Mi"
          limits:
            cpu: "200m"
            memory: "256Mi"
        restart_policy: OnFailure
        concurrency_policy: Forbid

    lifecycle:
      delete: false             # if true + previous_name set, deletes old resources
      delete_grace_seconds: 300

Secrets architecture

kforge works with three categories of secrets, each living in the right place for its scope.

Category A — Gitea org secrets

Set once at the organisation level; available to every repo automatically.

Secret Used by Purpose
DOCKER_USERNAME docker/login-action Registry authentication for image push
DOCKER_PASSWORD docker/login-action Registry authentication for image push
KFORGE_NODE_IP kforge Cluster node IP — written as the external-dns target annotation

DOCKER_USERNAME and DOCKER_PASSWORD are consumed by the Docker build steps in the generated workflow, not by kforge itself. kforge validate does not check for them.

Category B — Gitea repo secrets

Per-repo, since different apps may deploy to different clusters.

Secret Purpose
KUBE_HOST Kubernetes API server URL
KUBE_TOKEN Service account token
KUBE_CERTIFICATE Base64-encoded CA certificate

Category C — Cluster secrets (auto-generated)

Created by kforge secrets apply. Never appear in Gitea or in kforge.yml.

Secret name Contents
{full_name}-db-credentials PostgreSQL username + alphanumeric password
{full_name}-basic-auth htpasswd for ingress basic auth
{full_name}-cache-credentials Valkey/Redis password
{full_name}-storage-credentials Minio access/secret keys
{full_name}-queue-credentials RabbitMQ credentials
{full_name}-search-credentials Meilisearch master key

Run kforge secrets list at any time to see the full checklist with live status for your current repo.

Project structure

kforge.yml                        ← your app config (committed)
.gitea/
  workflows/
    deploy.yml                    ← generated by kforge gitea-actions (committed)
    preview.yml                   ← generated by kforge gitea-preview (committed, optional)

Generated manifests (.kforge-out/) are never committed — they are created at CI time and discarded after kubectl apply.

Planned

  • kforge serve — web UI for managing infrastructure across all repos
  • kforge lifecycle rename — interactive rename flow with resource patching
  • Route53 and Porkbun DNS providers
  • --set key=value flag for runtime overrides (e.g. --set image_tag=$SHA)
Reference in New Issue View Git Blame Copy Permalink