567 lines
17 KiB
Markdown
567 lines
17 KiB
Markdown
# kforge
|
|
|
|
**kforge** eliminates Kubernetes boilerplate. You define your app once in a `kforge.yml` file — environments, infrastructure, ingress, TLS, DNS — and kforge generates production-ready flat manifests on every CI run. Nothing is committed to your repo except the config.
|
|
|
|
Built for self-hosted MicroK8s, Gitea Actions, Cloudflare DNS, cert-manager, and CNPG — but designed to be extended.
|
|
|
|
---
|
|
|
|
## How it works
|
|
|
|
1. Add a `kforge.yml` to your repo root describing your app, environments, and infrastructure.
|
|
2. kforge generates Kubernetes YAML at CI time — Service, Deployment, Ingress, cert-manager Certificates, CronJobs, and infrastructure (database, cache, storage, queue, search).
|
|
3. Generated manifests are applied to the cluster and discarded. Only `kforge.yml` is committed.
|
|
|
|
```
|
|
kforge.yml → kforge generate → kubectl apply → cluster
|
|
```
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Option A — Build from source (recommended for self-hosted runners)
|
|
|
|
```bash
|
|
git clone https://gitea.yourdomain.com/yourorg/kforge.git
|
|
cd kforge
|
|
go build -o /usr/local/bin/kforge .
|
|
```
|
|
|
|
Install once on your Gitea runner host and it's available to every repo automatically.
|
|
|
|
### Option B — Gitea Package Registry
|
|
|
|
If you've published kforge to your Gitea instance's generic package registry:
|
|
|
|
```bash
|
|
curl -fsSL \
|
|
-H "Authorization: token $GITEA_TOKEN" \
|
|
"https://gitea.yourdomain.com/api/packages/yourorg/generic/kforge/latest/kforge-linux-amd64" \
|
|
-o /usr/local/bin/kforge
|
|
chmod +x /usr/local/bin/kforge
|
|
```
|
|
|
|
---
|
|
|
|
## Quick start
|
|
|
|
**1. Add `kforge.yml` to your repo:**
|
|
|
|
```yaml
|
|
meta:
|
|
name: my-app
|
|
tenant: my-org
|
|
|
|
registry:
|
|
url: registry.yourdomain.com
|
|
|
|
dns:
|
|
provider: cloudflare
|
|
cloudflare:
|
|
api_token: ${CLOUDFLARE_API_TOKEN}
|
|
zones:
|
|
- name: yourdomain.com
|
|
zone_id: ${CF_ZONE_ID_YOURDOMAIN}
|
|
node_ip: ${KFORGE_NODE_IP}
|
|
|
|
cluster:
|
|
tls_issuer: letsencrypt-prod
|
|
cnpg:
|
|
host: cnpg-main-rw.default.svc.cluster.local
|
|
|
|
defaults:
|
|
port: 3000
|
|
health_check:
|
|
path: /healthcheck
|
|
|
|
infrastructure:
|
|
database:
|
|
provider: cnpg
|
|
cache:
|
|
provider: valkey
|
|
mode: standalone
|
|
|
|
environments:
|
|
staging:
|
|
namespace: staging
|
|
image_tag: latest
|
|
env_vars:
|
|
- name: API_URL
|
|
value: https://api-staging.yourdomain.com
|
|
type: plain
|
|
ingress:
|
|
hosts:
|
|
- hostname: app-staging.yourdomain.com
|
|
tls: true
|
|
dns_record: true
|
|
auth:
|
|
enabled: true
|
|
users:
|
|
- yourname
|
|
|
|
production:
|
|
namespace: production
|
|
image_tag: latest
|
|
env_vars:
|
|
- name: API_URL
|
|
value: https://api.yourdomain.com
|
|
type: plain
|
|
ingress:
|
|
hosts:
|
|
- hostname: app.yourdomain.com
|
|
tls: true
|
|
dns_record: true
|
|
infrastructure:
|
|
cache:
|
|
mode: cluster
|
|
replicas: 3
|
|
```
|
|
|
|
**2. Validate your config and see what secrets you need:**
|
|
|
|
```bash
|
|
kforge validate
|
|
kforge secrets list
|
|
```
|
|
|
|
**3. Preview what gets generated:**
|
|
|
|
```bash
|
|
kforge generate --env production --dry-run
|
|
```
|
|
|
|
**4. Generate your Gitea Actions workflow:**
|
|
|
|
```bash
|
|
kforge gitea-actions > .gitea/workflows/deploy.yml
|
|
```
|
|
|
|
---
|
|
|
|
## CLI reference
|
|
|
|
### `kforge validate`
|
|
|
|
Parses `kforge.yml`, checks structural correctness, and verifies all required secrets are present in the current environment. Exits non-zero if anything is wrong — use this as the first step in CI to fail fast before touching the cluster.
|
|
|
|
```bash
|
|
kforge validate
|
|
kforge validate -c path/to/kforge.yml
|
|
```
|
|
|
|
---
|
|
|
|
### `kforge generate`
|
|
|
|
Generates flat Kubernetes manifests for one or all environments. Writes files to `.kforge-out/` by default, or prints to stdout with `--dry-run`.
|
|
|
|
```bash
|
|
kforge generate # all environments
|
|
kforge generate --env staging # one environment
|
|
kforge generate --env production --dry-run
|
|
kforge generate --env production --output .kube/
|
|
```
|
|
|
|
**Output files per environment:**
|
|
|
|
| File | Contents |
|
|
| --------------------------------- | ---------------------------------------------------- |
|
|
| `{env}-core.yaml` | Service, Deployment, Ingress, Certificates, CronJobs |
|
|
| `{env}-infra-cnpg-database.yaml` | CNPG Database CR |
|
|
| `{env}-infra-cnpg-role.yaml` | CNPG DatabaseRole CR |
|
|
| `{env}-infra-cache.yaml` | Valkey/Redis Deployment or StatefulSet |
|
|
| `{env}-infra-cache-svc.yaml` | Cache Service |
|
|
| `{env}-infra-storage.yaml` | Minio Deployment |
|
|
| `{env}-infra-queue-nats.yaml` | NATS Deployment |
|
|
| `{env}-infra-search.yaml` | Meilisearch Deployment |
|
|
| `{env}-infra-servicemonitor.yaml` | Prometheus ServiceMonitor CR |
|
|
|
|
Infrastructure env vars (`DATABASE_URL`, `CACHE_URL`, `STORAGE_ENDPOINT`, etc.) are automatically injected into the Deployment — you don't wire these up manually.
|
|
|
|
---
|
|
|
|
### `kforge secrets list`
|
|
|
|
Prints the full secrets checklist for this repo, grouped by where each secret needs to live, with live status checks against the current environment.
|
|
|
|
```bash
|
|
kforge secrets list
|
|
```
|
|
|
|
Example output:
|
|
|
|
```
|
|
── Gitea org secret ──
|
|
DOCKER_USERNAME ✗ missing
|
|
CLOUDFLARE_API_TOKEN ✓ set
|
|
CF_ZONE_ID_YOURDOMAIN_COM ✓ set
|
|
KFORGE_NODE_IP ✓ set
|
|
SOPS_AGE_KEY ✗ missing
|
|
|
|
── Gitea repo secret ──
|
|
KUBE_HOST ✓ set
|
|
KUBE_TOKEN ✓ set
|
|
KUBE_CERTIFICATE ✓ set
|
|
|
|
── Cluster secret (auto-generated) ──
|
|
prod-my-org-my-app-db-credentials — managed by kforge
|
|
prod-my-org-my-app-cache-credentials — managed by kforge
|
|
```
|
|
|
|
---
|
|
|
|
### `kforge secrets apply`
|
|
|
|
Generates secure random credentials and creates Kubernetes Secrets in the cluster for all enabled infrastructure services. Safe to run on every deploy — secrets are only created if they don't already exist. Use `--force` to rotate credentials.
|
|
|
|
```bash
|
|
kforge secrets apply --env staging
|
|
kforge secrets apply --env production --force # rotates all credentials
|
|
```
|
|
|
|
For basic auth secrets, kforge prints the generated passwords once at apply time. Save them — they are not stored anywhere else.
|
|
|
|
Requires `KUBE_HOST`, `KUBE_TOKEN`, and `KUBE_CERTIFICATE` to be set (Gitea injects these automatically during CI).
|
|
|
|
---
|
|
|
|
### `kforge dns ensure`
|
|
|
|
Creates or updates Cloudflare DNS A records for all ingress hosts with `dns_record: true`. Idempotent — no-ops if the record already points to the correct IP.
|
|
|
|
```bash
|
|
kforge dns ensure --env staging
|
|
kforge dns ensure --env staging --env production
|
|
```
|
|
|
|
Requires `CLOUDFLARE_API_TOKEN` and `KFORGE_NODE_IP` to be set.
|
|
|
|
---
|
|
|
|
### `kforge gitea-actions`
|
|
|
|
Generates a complete `.gitea/workflows/deploy.yml` for this app. Re-run whenever you add environments or change deploy configuration.
|
|
|
|
```bash
|
|
kforge gitea-actions
|
|
kforge gitea-actions --output .gitea/workflows/deploy.yml
|
|
kforge gitea-actions --branch main --env staging --env production
|
|
```
|
|
|
|
The generated workflow runs these steps for each environment, in order:
|
|
|
|
1. Build and push Docker image
|
|
2. `kforge validate`
|
|
3. `kforge secrets apply` — creates missing cluster secrets
|
|
4. `kforge dns ensure` — creates missing DNS records
|
|
5. `kforge generate` — writes manifests to `.kforge-out/`
|
|
6. `kubectl apply` — applies core manifests
|
|
7. `kubectl apply` — applies infra manifests
|
|
8. `kubectl rollout restart` — triggers rolling update
|
|
|
|
---
|
|
|
|
## `kforge.yml` reference
|
|
|
|
### Naming and interpolation
|
|
|
|
kforge generates resource names using the pattern `{env_prefix}-{tenant}-{name}` (e.g. `prod-my-org-my-app`). Use `${tokens}` anywhere in string values to reference resolved fields:
|
|
|
|
| Token | Resolves to |
|
|
| --------------- | ------------------------------------------- |
|
|
| `${name}` | `meta.name` |
|
|
| `${tenant}` | `meta.tenant` |
|
|
| `${env}` | current environment key |
|
|
| `${env_prefix}` | short env prefix (first 4 chars, or custom) |
|
|
| `${full_name}` | `{env_prefix}-{tenant}-{name}` |
|
|
| `${namespace}` | resolved namespace for the environment |
|
|
|
|
Environment variables (e.g. `${CLOUDFLARE_API_TOKEN}`) are resolved from the CI process environment at generation time — never hardcode secrets in `kforge.yml`.
|
|
|
|
---
|
|
|
|
### `meta`
|
|
|
|
```yaml
|
|
meta:
|
|
name: my-app # required — short app name, lowercase, hyphens ok
|
|
tenant: my-org # required — org/tenant identifier
|
|
name_override: ~ # optional — override the full generated resource name
|
|
previous_name: ~ # optional — set when renaming; kforge patches rather than recreates
|
|
```
|
|
|
|
---
|
|
|
|
### `registry`
|
|
|
|
```yaml
|
|
registry:
|
|
url: registry.yourdomain.com # default: registry.natelubitz.com
|
|
repository: my-org/my-app # default: {tenant}/{name}
|
|
pull_secret: regcred # default: regcred
|
|
```
|
|
|
|
Override per-environment by adding a `registry:` block under the environment.
|
|
|
|
---
|
|
|
|
### `dns`
|
|
|
|
```yaml
|
|
dns:
|
|
provider: cloudflare # currently supported: cloudflare
|
|
cloudflare:
|
|
api_token: ${CLOUDFLARE_API_TOKEN}
|
|
zones:
|
|
- name: yourdomain.com
|
|
zone_id: ${CF_ZONE_ID_YOURDOMAIN}
|
|
proxied: false # false = DNS-only, required for cert-manager DNS-01
|
|
node_ip: ${KFORGE_NODE_IP} # IP for new A records
|
|
skip_dns: false # true to disable all DNS management
|
|
```
|
|
|
|
kforge matches each ingress hostname to the correct zone by longest-suffix match — add one zone entry per domain you own.
|
|
|
|
---
|
|
|
|
### `cluster`
|
|
|
|
```yaml
|
|
cluster:
|
|
tls_issuer: letsencrypt-prod # default: letsencrypt-prod
|
|
ingress_class: nginx # default: nginx
|
|
cnpg:
|
|
host: cnpg-main-rw.default.svc.cluster.local # your CNPG cluster service
|
|
namespace_pattern: "${env}" # default: environment key
|
|
```
|
|
|
|
---
|
|
|
|
### `defaults`
|
|
|
|
All fields can be overridden per-environment.
|
|
|
|
```yaml
|
|
defaults:
|
|
port: 3000
|
|
replicas: 1
|
|
image_pull_policy: Always
|
|
service_type: ClusterIP
|
|
dockerfile: Dockerfile
|
|
health_check:
|
|
path: /healthcheck
|
|
port: 3000
|
|
initial_delay_seconds: 15
|
|
period_seconds: 10
|
|
timeout_seconds: 5
|
|
failure_threshold: 3
|
|
liveness: true
|
|
readiness: true
|
|
resources:
|
|
requests:
|
|
cpu: "100m"
|
|
memory: "128Mi"
|
|
limits:
|
|
cpu: "500m"
|
|
memory: "512Mi"
|
|
env_vars: []
|
|
```
|
|
|
|
---
|
|
|
|
### `infrastructure` (root defaults)
|
|
|
|
Define infrastructure at the root level and it applies to **all environments** by default. Per-environment blocks are a shallow merge — only the fields you specify override; everything else inherits.
|
|
|
|
```yaml
|
|
infrastructure:
|
|
database:
|
|
provider: cnpg # only supported provider currently
|
|
cache:
|
|
provider: valkey # valkey (recommended) | redis
|
|
mode: standalone # standalone | cluster
|
|
replicas: 1
|
|
storage:
|
|
enabled: false
|
|
provider: minio # standalone | distributed
|
|
queue:
|
|
enabled: false
|
|
provider: nats # nats (recommended, ~20MB) | rabbitmq (~200MB)
|
|
search:
|
|
enabled: false
|
|
provider: meilisearch
|
|
monitoring:
|
|
enabled: false
|
|
provider: prometheus
|
|
metrics_path: /metrics
|
|
```
|
|
|
|
If a service block exists at root, it is **enabled by default** for all environments. To disable it for a specific environment:
|
|
|
|
```yaml
|
|
environments:
|
|
dev:
|
|
infrastructure:
|
|
cache:
|
|
enabled: false
|
|
```
|
|
|
|
To scale up for production while staging uses the root defaults:
|
|
|
|
```yaml
|
|
environments:
|
|
production:
|
|
infrastructure:
|
|
cache:
|
|
mode: cluster
|
|
replicas: 3 # provider: valkey inherited from root
|
|
```
|
|
|
|
**Injected env vars per service** (automatically added to your Deployment):
|
|
|
|
| Service | Env vars injected |
|
|
| ---------------- | ------------------------------------------------------------------------- |
|
|
| database (CNPG) | `DATABASE_URL`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` |
|
|
| cache | `CACHE_URL`, `CACHE_HOST`, `CACHE_PORT`, `CACHE_PASSWORD` |
|
|
| storage | `STORAGE_ENDPOINT`, `STORAGE_ACCESS_KEY`, `STORAGE_SECRET_KEY` |
|
|
| queue (NATS) | `QUEUE_URL`, `QUEUE_HOST` |
|
|
| queue (RabbitMQ) | `QUEUE_URL`, `QUEUE_USER`, `QUEUE_PASSWORD` |
|
|
| search | `SEARCH_URL`, `SEARCH_MASTER_KEY` |
|
|
|
|
---
|
|
|
|
### `environments`
|
|
|
|
```yaml
|
|
environments:
|
|
production:
|
|
namespace: production
|
|
replicas: 1
|
|
image_tag: latest # override with --set image_tag=$SHA in CI
|
|
env_prefix: prod # default: first 4 chars of env key
|
|
|
|
env_vars:
|
|
- name: API_URL
|
|
type: plain
|
|
value: https://api.yourdomain.com
|
|
|
|
- name: SOME_SECRET
|
|
type: secret_ref # pull from an existing Kubernetes Secret
|
|
secret_name: my-secrets
|
|
secret_key: some_secret
|
|
|
|
- name: FEATURE_FLAG
|
|
type: configmap_ref # pull from a ConfigMap
|
|
configmap_name: my-config
|
|
configmap_key: feature_flag
|
|
|
|
ingress:
|
|
hosts:
|
|
- hostname: app.yourdomain.com
|
|
tls: true # kforge generates a cert-manager Certificate
|
|
dns_record: true # kforge creates a Cloudflare A record
|
|
auth:
|
|
enabled: false # enable for staging/dev to protect unreleased work
|
|
users:
|
|
- yourname # passwords are auto-generated by kforge secrets apply
|
|
|
|
infrastructure:
|
|
# shallow merge on top of root — only override what differs
|
|
cache:
|
|
mode: cluster
|
|
replicas: 3
|
|
|
|
cron_jobs:
|
|
- name: cleanup
|
|
schedule: "0 2 * * *"
|
|
command: ["node", "scripts/cleanup.js"]
|
|
inherit_env: true # inherits all deployment env vars
|
|
env_vars:
|
|
- name: BATCH_SIZE
|
|
value: "500"
|
|
type: plain
|
|
resources:
|
|
requests:
|
|
cpu: "50m"
|
|
memory: "64Mi"
|
|
limits:
|
|
cpu: "200m"
|
|
memory: "256Mi"
|
|
restart_policy: OnFailure
|
|
concurrency_policy: Forbid
|
|
|
|
lifecycle:
|
|
delete: false # if true + previous_name set, deletes old resources
|
|
delete_grace_seconds: 300 # 5-minute countdown before deletion runs in CI
|
|
```
|
|
|
|
---
|
|
|
|
## Secrets architecture
|
|
|
|
kforge works with three categories of secrets, each living in the right place for its scope.
|
|
|
|
### Category A — Gitea org secrets
|
|
|
|
Set once at the organisation level; available to every repo automatically.
|
|
|
|
| Secret | Purpose |
|
|
| ---------------------- | ---------------------------------------------- |
|
|
| `DOCKER_USERNAME` | Registry authentication |
|
|
| `DOCKER_PASSWORD` | Registry authentication |
|
|
| `CLOUDFLARE_API_TOKEN` | DNS record management (Zone:Read + DNS:Edit) |
|
|
| `CF_ZONE_ID_{DOMAIN}` | One per zone, e.g. `CF_ZONE_ID_YOURDOMAIN_COM` |
|
|
| `KFORGE_NODE_IP` | MicroK8s node IP for DNS A records |
|
|
| `SOPS_AGE_KEY` | Decrypts `.kforge/secrets.enc.yml` |
|
|
|
|
### Category B — Gitea repo secrets
|
|
|
|
Per-repo, since different apps may deploy to different clusters.
|
|
|
|
| Secret | Purpose |
|
|
| ------------------ | ----------------------------- |
|
|
| `KUBE_HOST` | Kubernetes API server URL |
|
|
| `KUBE_TOKEN` | Service account token |
|
|
| `KUBE_CERTIFICATE` | Base64-encoded CA certificate |
|
|
|
|
### Category C — Cluster secrets (auto-generated)
|
|
|
|
Created by `kforge secrets apply`. Never appear in Gitea or in `kforge.yml`.
|
|
|
|
| Secret name | Contents |
|
|
| --------------------------------- | --------------------------------- |
|
|
| `{full_name}-db-credentials` | CNPG-managed database credentials |
|
|
| `{full_name}-basic-auth` | htpasswd for ingress basic auth |
|
|
| `{full_name}-cache-credentials` | Valkey/Redis password |
|
|
| `{full_name}-storage-credentials` | Minio access/secret keys |
|
|
| `{full_name}-queue-credentials` | RabbitMQ credentials |
|
|
| `{full_name}-search-credentials` | Meilisearch master key |
|
|
|
|
Run `kforge secrets list` at any time to see the full checklist with live status for your current repo.
|
|
|
|
---
|
|
|
|
## Project structure
|
|
|
|
```
|
|
kforge.yml ← your app config (committed)
|
|
.gitea/
|
|
workflows/
|
|
deploy.yml ← generated by kforge gitea-actions (committed)
|
|
.kforge/
|
|
secrets.enc.yml ← SOPS-encrypted sensitive config (committed)
|
|
kforge.age ← age private key (NEVER committed — goes in Gitea as SOPS_AGE_KEY)
|
|
```
|
|
|
|
Generated manifests (`.kforge-out/`) are never committed — they are created at CI time and discarded after `kubectl apply`.
|
|
|
|
---
|
|
|
|
## Planned
|
|
|
|
- `kforge serve` — web UI for managing infrastructure across all repos
|
|
- `kforge lifecycle rename` — interactive rename flow with resource patching
|
|
- Route53 and Porkbun DNS providers
|
|
- `--set key=value` flag for runtime overrides (e.g. `--set image_tag=$SHA`)
|