12 Commits

Author SHA1 Message Date
nate.lubitz 9596622c58 update to public url
Publish Action Image / publish (push) Successful in 38s
2026-06-29 18:06:08 +10:00
nate.lubitz d4ac9ee361 update yml
Publish Action Image / publish (push) Failing after 3s
2026-06-29 17:58:13 +10:00
nate.lubitz ca88bccadf remove docker login
Publish Action Image / build (push) Failing after 4s
Publish Action Image / publish (push) Failing after 13s
2026-06-29 17:46:30 +10:00
nate.lubitz b9727afeab more updates
Publish Action Image / build (push) Failing after 32s
Publish Action Image / publish (push) Failing after 52s
2026-06-29 17:44:29 +10:00
nate.lubitz f323d18cf7 insecure registry
Publish Action Image / build (push) Failing after 9s
Publish Action Image / publish (push) Failing after 9s
2026-06-29 17:32:29 +10:00
nate.lubitz 52c986b859 update registry url
Publish Action Image / build (push) Failing after 6s
Publish Action Image / publish (push) Failing after 2m7s
2026-06-29 17:16:44 +10:00
nate.lubitz 347bfb4df0 allow insecure piece
Publish Action Image / build (push) Failing after 8s
Publish Action Image / publish (push) Failing after 10s
2026-06-29 17:14:59 +10:00
nate.lubitz 565a91e235 update validation
Publish Action Image / build (push) Successful in 33s
Publish Action Image / publish (push) Successful in 28s
2026-06-29 16:58:52 +10:00
nate.lubitz 1881ea2e99 more updates
Publish Action Image / build (push) Successful in 24s
Publish Action Image / publish (push) Successful in 27s
2026-06-29 16:29:16 +10:00
nate.lubitz 8a2225bf08 another attempt
Publish Action Image / build (push) Successful in 24s
Publish Action Image / publish (push) Successful in 25s
2026-06-29 16:07:46 +10:00
nate.lubitz 9adb5780f2 use as gitea action
Publish Action Image / build (push) Successful in 31s
2026-06-29 16:01:01 +10:00
nate.lubitz 4ee1b9e13c more updates
Publish Action Image / build (push) Successful in 1m8s
2026-06-29 15:14:55 +10:00
23 changed files with 1451 additions and 1083 deletions
-24
View File
@@ -1,24 +0,0 @@
# .gitea/workflows/publish.yml
name: Publish Action Image
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Login to registry
uses: docker/login-action@v2
with:
registry: registry.natelubitz.com
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and push action image
run: |
docker build -t registry.natelubitz.com/infra/kforge:latest .
docker push registry.natelubitz.com/infra/kforge:latest
+34
View File
@@ -0,0 +1,34 @@
# Builds and pushes the kforge Docker action image on every push to main.
# Other repos reference this image via:
# action_ref: registry.container-registry.svc.cluster.local:5000/infra/kforge:latest
# in their kforge.yml, which generates:
# uses: docker://registry.container-registry.svc.cluster.local:5000/infra/kforge:latest
name: Publish Action Image
on:
push:
branches:
- main
jobs:
publish:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Login to registry
uses: docker/login-action@v2
with:
password: ${{ secrets.DOCKER_PASSWORD }}
registry: registry.natelubitz.com
username: ${{ secrets.DOCKER_USERNAME }}
- name: Build and push kforge action image
env:
DOCKER_BUILDKIT: "0"
run: |
docker build -t registry.natelubitz.com/infra/kforge:latest .
docker push registry.natelubitz.com/infra/kforge:latest
+167
View File
@@ -0,0 +1,167 @@
# kforge
**kforge** generates production-ready Kubernetes manifests from a single `kforge.yml` in your repository root. Designed for self-hosted homelab deployments on MicroK8s with Gitea Actions CI/CD.
## Core concept
```
kforge.yml → kforge generate → kubectl apply → cluster
```
Only `kforge.yml` and generated workflow files are committed. Generated manifests are applied and discarded each CI run.
## Build and test
```sh
go build -o kforge .
go test ./...
```
## Commands
| Command | Purpose |
|---|---|
| `kforge validate` | Validate kforge.yml, list required secrets |
| `kforge generate [--env E]` | Generate Kubernetes manifests to `.kforge-out/` |
| `kforge generate --pr-number N` | Generate manifests for a PR preview environment |
| `kforge secrets apply --env E` | Generate and apply random credentials to cluster |
| `kforge secrets apply --pr-number N` | Apply credentials for a PR preview environment |
| `kforge gitea-actions` | Generate `.gitea/workflows/deploy.yml` |
| `kforge gitea-preview` | Generate `.gitea/workflows/preview.yml` for PR previews |
## Architecture
### Config system (`internal/config/`)
- `types.go` — all config structs mirroring `kforge.yml`
- `loader.go` — YAML parsing and structural validation
- `defaults.go``ApplyDefaults()`, `ResolveEnvironment()`, `SynthesizePreviewEnvironment()`
### Generator system (`internal/generator/`)
- `manifests.go` — Service, Deployment, Ingress (with external-dns annotations), Certificate, CronJob
- `infrastructure.go` — CNPG Database CR + db-init Job, Valkey, Minio, NATS, Meilisearch
- `gitea_actions.go` — deploy workflow (`GenerateGiteaActions`) + preview workflow (`GeneratePreviewActions`)
### CLI (`cmd/`)
- Uses [Cobra](https://github.com/spf13/cobra) for subcommands
- `root.go` — shared `loadConfig()` helper
- Each command calls `loadConfig()` then delegates to generators
### Token interpolation (`pkg/interpolate/`)
- Built-in: `${name}`, `${tenant}`, `${env}`, `${env_prefix}`, `${full_name}`, `${namespace}`, `${image_tag}`
- Falls back to `os.Getenv()` — CI secrets (e.g. `${KFORGE_NODE_IP}`) are injected this way
- `PGIdentifier()` converts a kforge name to a valid unquoted PostgreSQL identifier
## DNS and TLS
DNS is managed by **external-dns** running in the cluster — no Cloudflare API calls from kforge.
When a host has `dns_record: true`, kforge adds to the Ingress:
```yaml
annotations:
external-dns.alpha.kubernetes.io/hostname: "app.example.com"
external-dns.alpha.kubernetes.io/target: "<dns.target>"
```
`dns.target` in `kforge.yml` sets the target value (your node IP or a static hostname). Supports `${KFORGE_NODE_IP}` token which is resolved from the CI environment.
TLS is managed by **cert-manager** via a `ClusterIssuer`. A `Certificate` CR is generated for each host with `tls: true`. No separate DNS step needed in the workflow.
## CNPG database (centralized cluster)
For a centralized CNPG cluster, kforge generates:
1. A CNPG `Database` CR — declaratively manages the database lifecycle
2. A `db-init` Kubernetes Job — creates the PostgreSQL role and syncs its password on every deploy
`kforge secrets apply` creates a `${full_name}-db-credentials` Secret containing an alphanumeric `username` and random `password` **before** the Job runs. The password is alphanumeric-only so it can be safely used in shell commands within the Job.
Required cluster resources:
- CNPG superuser Secret named by `cluster.cnpg.superuser_secret` (default: `cnpg-main-superuser`)
- CNPG cluster named by `cluster.cnpg.cluster_name` (default: `cnpg-main`)
## PR Preview environments
Add a `preview:` block to `kforge.yml` and run `kforge gitea-preview` to generate `.gitea/workflows/preview.yml`.
The preview workflow:
- **PR open / sync**: Creates namespace `preview-pr-{N}`, builds image tagged `:pr-{N}`, deploys app
- **PR close**: Deletes the namespace (removes all preview resources)
`kforge generate --pr-number N` and `kforge secrets apply --pr-number N` synthesize the preview environment at runtime using `preview.base_environment` settings with namespace/hostname overrides.
A `Namespace` manifest is included in the generated output so `kubectl apply` creates it automatically.
## Required secrets (Gitea)
### Org-level
- `DOCKER_USERNAME`, `DOCKER_PASSWORD` — registry auth
- `KFORGE_NODE_IP` — cluster node IP (used as external-dns target)
### Repo-level
- `KUBE_HOST`, `KUBE_TOKEN`, `KUBE_CERTIFICATE` — kubectl auth
## kforge.yml reference
```yaml
meta:
name: my-app
tenant: my-tenant
registry:
url: registry.example.com
pull_secret: regcred # default
dns:
target: ${KFORGE_NODE_IP} # external-dns annotation target
cluster:
tls_issuer: letsencrypt-prod # default
ingress_class: nginx # default
cnpg:
host: cnpg-main-rw.default.svc.cluster.local # default
cluster_name: cnpg-main # default
superuser_secret: cnpg-main-superuser # default
defaults:
port: 3000
health_check:
path: /healthcheck
resources:
requests: { cpu: 100m, memory: 128Mi }
limits: { cpu: 500m, memory: 512Mi }
infrastructure:
database:
provider: cnpg
cache:
provider: valkey
mode: standalone
preview:
enabled: true
base_environment: staging
namespace_prefix: preview-pr # namespace = preview-pr-{N}
hostname_template: "pr-${PR_NUMBER}.${name}.example.com"
environments:
staging:
namespace: staging
image_tag: latest
ingress:
hosts:
- hostname: app-staging.example.com
tls: true
dns_record: true
production:
namespace: production
image_tag: latest
ingress:
hosts:
- hostname: app.example.com
tls: true
dns_record: true
infrastructure:
cache:
mode: cluster
replicas: 3
```
+11 -8
View File
@@ -1,15 +1,18 @@
FROM golang:1.22-alpine AS builder FROM golang:1.22-alpine AS builder
WORKDIR /app WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . . COPY . .
RUN go build -o kforge . RUN go build -o /usr/local/bin/kforge .
FROM alpine:3.19 FROM alpine:3.20
COPY --from=builder /app/kforge /usr/local/bin/kforge RUN apk add --no-cache ca-certificates curl git
RUN apk add --no-cache curl docker-cli && \
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && \
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl && \
curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin
ARG KUBECTL_VERSION=v1.31.0
RUN curl -fsSL "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl" \
-o /usr/local/bin/kubectl && chmod +x /usr/local/bin/kubectl
COPY --from=builder /usr/local/bin/kforge /usr/local/bin/kforge
COPY entrypoint.sh /entrypoint.sh COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh RUN chmod +x /entrypoint.sh
+19 -52
View File
@@ -1,62 +1,29 @@
name: "K8s YAML Generator" name: 'kforge'
description: "Builds a Docker image, pushes it to a private registry, generates Kubernetes YAML from a simplified YML file, and deploys it." description: 'Generate and apply Kubernetes manifests from kforge.yml'
author: "Claude Code made this"
inputs: inputs:
image_name: command:
description: "Docker image name to build and push (e.g. my-app)" description: 'deploy | preview-up | preview-down | validate | secrets'
required: true
image_tag:
description: "Docker image tag. If omitted, defaults to both 'latest' and the short commit SHA."
required: false required: false
dockerfile: default: 'deploy'
description: "Path to Dockerfile" env:
description: 'Environment to target (e.g. production, staging). Omit to target all.'
required: false required: false
default: "Dockerfile" config:
max_tags: description: 'Path to kforge.yml relative to the workspace root'
description: "Maximum number of SHA image tags to keep in the registry"
required: false required: false
default: "5" default: 'kforge.yml'
namespace:
registry: description: 'Kubernetes namespace for rollout restart (defaults to env name)'
description: "Docker registry URL"
required: false required: false
default: "registry.natelubitz.com" pr_number:
registry_username: description: 'PR number — required for preview-up and preview-down'
description: "Registry username"
required: true
registry_password:
description: "Registry password"
required: true
kube_host:
description: "Kubernetes API server URL"
required: false required: false
default: "192.168.1.20:16443" namespace_prefix:
kube_certificate: description: 'Namespace prefix for preview environments'
description: "Base64 encoded Kubernetes CA certificate"
required: true
kube_token:
description: "Kubernetes service account token"
required: true
scan_image:
description: "Scan image for vulnerabilities before pushing"
required: false required: false
default: "true" default: 'preview-pr'
scan_severity:
description: "Fail on these severity levels (UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL)"
required: false
default: "HIGH,CRITICAL"
# outputs:
# output_file:
# description: "Path to the generated Kubernetes YAML file"
runs: runs:
using: "docker" using: docker
image: "docker://registry.natelubitz.com/infra/kforge:latest" image: Dockerfile
# args:
# - ${{ inputs.input_file }}
# - ${{ inputs.output_file }}
# - ${{ inputs.auto_deploy }}
+71 -89
View File
@@ -4,88 +4,11 @@ import (
"fmt" "fmt"
"os" "os"
"kforge/internal/config"
dnsProvider "kforge/internal/dns"
"kforge/internal/generator" "kforge/internal/generator"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
// ------------------------------------------------------------
// kforge dns ensure
// ------------------------------------------------------------
var dnsCmd = &cobra.Command{
Use: "dns",
Short: "Manage DNS records for kforge environments",
}
var dnsEnsureEnvs []string
var dnsEnsureCmd = &cobra.Command{
Use: "ensure",
Short: "Create or update DNS A records for ingress hosts",
Long: `For each ingress host with dns_record: true, creates or updates
an A record pointing to KFORGE_NODE_IP.
Idempotent — safe to run on every deploy. Skips hosts where the
record already points to the correct IP.
Examples:
kforge dns ensure --env staging
kforge dns ensure --env staging --env production`,
RunE: runDNSEnsure,
}
func init() {
dnsEnsureCmd.Flags().StringArrayVarP(&dnsEnsureEnvs, "env", "e", nil,
"Environment(s) to ensure DNS records for (default: all)")
dnsCmd.AddCommand(dnsEnsureCmd)
rootCmd.AddCommand(dnsCmd)
}
func runDNSEnsure(cmd *cobra.Command, args []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
if cfg.DNS.SkipDNS {
fmt.Println("dns.skip_dns is true — skipping DNS management")
return nil
}
nodeIP := cfg.DNS.NodeIP
if nodeIP == "" {
nodeIP = os.Getenv("KFORGE_NODE_IP")
}
if nodeIP == "" {
return fmt.Errorf("node IP not set: add dns.node_ip to kforge.yml or set KFORGE_NODE_IP")
}
provider, err := dnsProvider.NewProvider(cfg.DNS)
if err != nil {
return fmt.Errorf("initialising DNS provider: %w", err)
}
envKeys := dnsEnsureEnvs
if len(envKeys) == 0 {
envKeys = config.EnvironmentKeys(cfg)
}
for _, envKey := range envKeys {
fmt.Printf("\nEnsuring DNS records for: %s\n", envKey)
env, err := config.ResolveEnvironment(cfg, envKey)
if err != nil {
return err
}
if err := dnsProvider.EnsureRecordsForEnvironment(provider, &env, nodeIP); err != nil {
return fmt.Errorf("env %q: %w", envKey, err)
}
}
return nil
}
// ------------------------------------------------------------ // ------------------------------------------------------------
// kforge gitea-actions // kforge gitea-actions
// ------------------------------------------------------------ // ------------------------------------------------------------
@@ -98,16 +21,17 @@ var (
var giteaActionsCmd = &cobra.Command{ var giteaActionsCmd = &cobra.Command{
Use: "gitea-actions", Use: "gitea-actions",
Short: "Generate a Gitea Actions workflow for this app", Short: "Generate a Gitea Actions deploy workflow for this app",
Long: `Generates a complete .gitea/workflows/deploy.yml that: Long: `Generates .gitea/workflows/deploy.yml that:
- Builds and pushes the Docker image on every push to main - Builds and pushes the Docker image on push to main
- Runs kforge validate - Runs kforge validate
- Applies cluster secrets (idempotent) - Applies cluster secrets (idempotent)
- Ensures DNS records
- Generates manifests and applies them with kubectl - Generates manifests and applies them with kubectl
- Rolls out the deployment - Rolls out the deployment
The generated workflow replaces your hand-written deploy.yml. DNS is handled automatically by external-dns reading the Ingress
annotations that kforge writes — no separate DNS step needed.
Re-run whenever you add environments or change deploy options. Re-run whenever you add environments or change deploy options.
Examples: Examples:
@@ -142,29 +66,87 @@ func runGiteaActions(cmd *cobra.Command, args []string) error {
return fmt.Errorf("generating workflow: %w", err) return fmt.Errorf("generating workflow: %w", err)
} }
if giteaActionsOutput == "-" { return writeWorkflow(giteaActionsOutput, workflow)
fmt.Print(workflow) }
// ------------------------------------------------------------
// kforge gitea-preview
// ------------------------------------------------------------
var giteaPreviewOutput string
var giteaPreviewCmd = &cobra.Command{
Use: "gitea-preview",
Short: "Generate a Gitea Actions workflow for PR preview environments",
Long: `Generates .gitea/workflows/preview.yml that:
- On PR open/sync: builds a PR-tagged image, applies secrets,
generates manifests (including a Namespace), and deploys.
- On PR close: deletes the preview namespace and all resources.
Requires preview.enabled: true in kforge.yml.
Example kforge.yml preview block:
preview:
enabled: true
base_environment: staging
namespace_prefix: preview-pr
hostname_template: "pr-${PR_NUMBER}.${name}.example.com"
Examples:
kforge gitea-preview
kforge gitea-preview --output .gitea/workflows/preview.yml`,
RunE: runGiteaPreview,
}
func init() {
giteaPreviewCmd.Flags().StringVarP(&giteaPreviewOutput, "output", "o",
".gitea/workflows/preview.yml",
"Output path for the generated preview workflow file")
rootCmd.AddCommand(giteaPreviewCmd)
}
func runGiteaPreview(cmd *cobra.Command, args []string) error {
cfg, err := loadConfig()
if err != nil {
return err
}
workflow, err := generator.GeneratePreviewActions(cfg)
if err != nil {
return fmt.Errorf("generating preview workflow: %w", err)
}
return writeWorkflow(giteaPreviewOutput, workflow)
}
// ------------------------------------------------------------
// Shared helpers
// ------------------------------------------------------------
func writeWorkflow(path, content string) error {
if path == "-" {
fmt.Print(content)
return nil return nil
} }
// Ensure parent directory exists. // Ensure parent directory exists.
dir := giteaActionsOutput dir := path
for i := len(dir) - 1; i >= 0; i-- { for i := len(dir) - 1; i >= 0; i-- {
if dir[i] == '/' { if dir[i] == '/' || dir[i] == '\\' {
dir = dir[:i] dir = dir[:i]
break break
} }
} }
if dir != giteaActionsOutput { if dir != path {
if err := os.MkdirAll(dir, 0o755); err != nil { if err := os.MkdirAll(dir, 0o755); err != nil {
return fmt.Errorf("creating output directory: %w", err) return fmt.Errorf("creating output directory: %w", err)
} }
} }
if err := os.WriteFile(giteaActionsOutput, []byte(workflow), 0o644); err != nil { if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
return fmt.Errorf("writing workflow: %w", err) return fmt.Errorf("writing workflow: %w", err)
} }
fmt.Printf("✓ Gitea Actions workflow written to %s\n", giteaActionsOutput) fmt.Printf("✓ Workflow written to %s\n", path)
return nil return nil
} }
+102 -29
View File
@@ -16,24 +16,29 @@ var (
generateEnvs []string generateEnvs []string
generateOutput string generateOutput string
generateDry bool generateDry bool
generatePRNumber string
) )
var generateCmd = &cobra.Command{ var generateCmd = &cobra.Command{
Use: "generate", Use: "generate",
Short: "Generate Kubernetes manifests from kforge.yml", Short: "Generate Kubernetes manifests from kforge.yml",
Long: `Reads kforge.yml (or the file specified with --config), resolves Long: `Reads kforge.yml, resolves each requested environment, and writes
each requested environment, and writes flat Kubernetes manifest flat Kubernetes manifest files to the output directory.
files to the output directory.
If no --env flags are given, manifests are generated for all If no --env flags are given, manifests are generated for all
environments defined in kforge.yml. environments defined in kforge.yml.
Use --pr-number to generate manifests for a PR preview environment.
A Namespace manifest is prepended so kubectl apply creates the
namespace automatically.
Examples: Examples:
kforge generate kforge generate
kforge generate --env staging kforge generate --env staging
kforge generate --env staging --env production kforge generate --env staging --env production
kforge generate --env production --output .kube/ kforge generate --env production --output .kube/
kforge generate --dry-run`, kforge generate --dry-run
kforge generate --pr-number 42`,
RunE: runGenerate, RunE: runGenerate,
} }
@@ -44,6 +49,8 @@ func init() {
"Directory to write generated manifests into") "Directory to write generated manifests into")
generateCmd.Flags().BoolVar(&generateDry, "dry-run", false, generateCmd.Flags().BoolVar(&generateDry, "dry-run", false,
"Print manifests to stdout instead of writing files") "Print manifests to stdout instead of writing files")
generateCmd.Flags().StringVar(&generatePRNumber, "pr-number", "",
"PR number — synthesizes and generates a preview environment")
rootCmd.AddCommand(generateCmd) rootCmd.AddCommand(generateCmd)
} }
@@ -53,6 +60,15 @@ func runGenerate(cmd *cobra.Command, args []string) error {
return err return err
} }
// Preview mode: synthesize the preview environment.
if generatePRNumber != "" {
env, err := config.SynthesizePreviewEnvironment(cfg, generatePRNumber)
if err != nil {
return fmt.Errorf("synthesizing preview environment: %w", err)
}
return generateForPreview(cfg, &env, generatePRNumber)
}
envKeys := generateEnvs envKeys := generateEnvs
if len(envKeys) == 0 { if len(envKeys) == 0 {
envKeys = config.EnvironmentKeys(cfg) envKeys = config.EnvironmentKeys(cfg)
@@ -78,30 +94,9 @@ func generateForEnv(cfg *config.KforgeConfig, envKey string) error {
return err return err
} }
// Core manifests (Service, Deployment, Ingress, Certs). coreYAML, infraManifests, err := buildManifests(cfg, &env)
coreYAML, err := generator.GenerateAll(&env, cfg)
if err != nil { if err != nil {
return fmt.Errorf("generating core manifests: %w", err) return err
}
// Infrastructure manifests.
infraManifests, err := generator.GenerateInfrastructure(&env)
if err != nil {
return fmt.Errorf("generating infrastructure manifests: %w", err)
}
// Collect all infrastructure env vars and append to deployment.
// We re-generate core manifests after injecting infra env vars.
var infraEnvVars []config.EnvVarConfig
for _, m := range infraManifests {
infraEnvVars = append(infraEnvVars, m.EnvVars...)
}
if len(infraEnvVars) > 0 {
env.EnvVars = config.MergeEnvVars(env.EnvVars, infraEnvVars)
coreYAML, err = generator.GenerateAll(&env, cfg)
if err != nil {
return fmt.Errorf("re-generating core manifests with infra vars: %w", err)
}
} }
if generateDry { if generateDry {
@@ -112,14 +107,16 @@ func generateForEnv(cfg *config.KforgeConfig, envKey string) error {
return nil return nil
} }
// Write core manifest. if err := os.MkdirAll(generateOutput, 0o755); err != nil {
return fmt.Errorf("creating output directory: %w", err)
}
coreFile := filepath.Join(generateOutput, envKey+"-core.yaml") coreFile := filepath.Join(generateOutput, envKey+"-core.yaml")
if err := os.WriteFile(coreFile, []byte(coreYAML), 0o644); err != nil { if err := os.WriteFile(coreFile, []byte(coreYAML), 0o644); err != nil {
return fmt.Errorf("writing core manifest: %w", err) return fmt.Errorf("writing core manifest: %w", err)
} }
fmt.Printf(" ✓ %s\n", coreFile) fmt.Printf(" ✓ %s\n", coreFile)
// Write infra manifests.
for _, m := range infraManifests { for _, m := range infraManifests {
infraFile := filepath.Join(generateOutput, envKey+"-infra-"+m.Name+".yaml") infraFile := filepath.Join(generateOutput, envKey+"-infra-"+m.Name+".yaml")
if err := os.WriteFile(infraFile, []byte(m.Content), 0o644); err != nil { if err := os.WriteFile(infraFile, []byte(m.Content), 0o644); err != nil {
@@ -131,6 +128,82 @@ func generateForEnv(cfg *config.KforgeConfig, envKey string) error {
return nil return nil
} }
// generateForPreview generates manifests for a synthesized preview
// environment, prepending a Namespace manifest so kubectl apply
// creates the namespace in a single pass.
func generateForPreview(cfg *config.KforgeConfig, env *config.ResolvedEnvironment, prNumber string) error {
coreYAML, infraManifests, err := buildManifests(cfg, env)
if err != nil {
return err
}
// Prepend a Namespace manifest so kubectl apply is self-contained.
nsYAML := generator.Namespace(env.Namespace, map[string]string{
"managed-by": "kforge",
"kforge/preview": "true",
"kforge/pr": prNumber,
})
coreYAML = nsYAML + generator.Separator + coreYAML
envKey := "preview-" + prNumber
if generateDry {
printDryRun(envKey, "core", coreYAML)
for _, m := range infraManifests {
printDryRun(envKey, m.Name, m.Content)
}
return nil
}
if err := os.MkdirAll(generateOutput, 0o755); err != nil {
return fmt.Errorf("creating output directory: %w", err)
}
coreFile := filepath.Join(generateOutput, envKey+"-core.yaml")
if err := os.WriteFile(coreFile, []byte(coreYAML), 0o644); err != nil {
return fmt.Errorf("writing core manifest: %w", err)
}
fmt.Printf(" ✓ %s\n", coreFile)
for _, m := range infraManifests {
infraFile := filepath.Join(generateOutput, envKey+"-infra-"+m.Name+".yaml")
if err := os.WriteFile(infraFile, []byte(m.Content), 0o644); err != nil {
return fmt.Errorf("writing %s manifest: %w", m.Name, err)
}
fmt.Printf(" ✓ %s\n", infraFile)
}
return nil
}
// buildManifests generates core + infra manifests for a resolved environment,
// injecting infra env vars into the deployment.
func buildManifests(cfg *config.KforgeConfig, env *config.ResolvedEnvironment) (string, []generator.InfraManifest, error) {
coreYAML, err := generator.GenerateAll(env, cfg)
if err != nil {
return "", nil, fmt.Errorf("generating core manifests: %w", err)
}
infraManifests, err := generator.GenerateInfrastructure(env)
if err != nil {
return "", nil, fmt.Errorf("generating infrastructure manifests: %w", err)
}
var infraEnvVars []config.EnvVarConfig
for _, m := range infraManifests {
infraEnvVars = append(infraEnvVars, m.EnvVars...)
}
if len(infraEnvVars) > 0 {
env.EnvVars = config.MergeEnvVars(env.EnvVars, infraEnvVars)
coreYAML, err = generator.GenerateAll(env, cfg)
if err != nil {
return "", nil, fmt.Errorf("re-generating core manifests with infra vars: %w", err)
}
}
return coreYAML, infraManifests, nil
}
func printDryRun(envKey, name, content string) { func printDryRun(envKey, name, content string) {
fmt.Printf("\n%s\n# --- %s / %s ---\n%s\n", fmt.Printf("\n%s\n# --- %s / %s ---\n%s\n",
generator.Separator, generator.Separator,
+52 -41
View File
@@ -10,17 +10,20 @@ import (
"strings" "strings"
"kforge/internal/config" "kforge/internal/config"
"kforge/pkg/interpolate"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
// passwordChars mirrors the character set from your original // passwordChars is the character set for general-purpose passwords.
// shell command: A-Za-z0-9 + printable special chars. // Database passwords use alphanumeric-only (see generateAlphanumeric)
// so they are safe to embed in shell commands inside the db-init Job.
const passwordChars = `ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!"#$%&'()*+,-./:;<=>?@[\]^_{|}~` const passwordChars = `ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!"#$%&'()*+,-./:;<=>?@[\]^_{|}~`
var ( var (
secretsApplyEnvs []string secretsApplyEnvs []string
secretsApplyForce bool secretsApplyForce bool
secretsApplyPRNumber string
) )
var secretsApplyCmd = &cobra.Command{ var secretsApplyCmd = &cobra.Command{
@@ -30,24 +33,25 @@ var secretsApplyCmd = &cobra.Command{
services and creates/updates Kubernetes Secrets in the cluster. services and creates/updates Kubernetes Secrets in the cluster.
Secrets are created with kubectl — KUBE_HOST, KUBE_TOKEN, and Secrets are created with kubectl — KUBE_HOST, KUBE_TOKEN, and
KUBE_CERTIFICATE must be set in the environment (Gitea injects KUBE_CERTIFICATE must be set in the environment.
these automatically during CI runs).
Already-existing secrets are NOT overwritten unless --force is Already-existing secrets are NOT overwritten unless --force is
passed. This prevents accidental credential rotation. passed. This prevents accidental credential rotation.
Examples: Examples:
kforge secrets apply --env staging kforge secrets apply --env staging
kforge secrets apply --env production --force`, kforge secrets apply --env production --force
kforge secrets apply --pr-number 42`,
RunE: runSecretsApply, RunE: runSecretsApply,
} }
func init() { func init() {
secretsApplyCmd.Flags().StringArrayVarP(&secretsApplyEnvs, "env", "e", nil, secretsApplyCmd.Flags().StringArrayVarP(&secretsApplyEnvs, "env", "e", nil,
"Environment(s) to apply secrets for (required)") "Environment(s) to apply secrets for")
secretsApplyCmd.Flags().BoolVar(&secretsApplyForce, "force", false, secretsApplyCmd.Flags().BoolVar(&secretsApplyForce, "force", false,
"Overwrite existing secrets (triggers credential rotation)") "Overwrite existing secrets (triggers credential rotation)")
_ = secretsApplyCmd.MarkFlagRequired("env") secretsApplyCmd.Flags().StringVar(&secretsApplyPRNumber, "pr-number", "",
"PR number — synthesizes a preview environment instead of a named env")
secretsCmd.AddCommand(secretsApplyCmd) secretsCmd.AddCommand(secretsApplyCmd)
} }
@@ -57,30 +61,58 @@ func runSecretsApply(cmd *cobra.Command, args []string) error {
return err return err
} }
// Preview mode: synthesize the preview environment.
if secretsApplyPRNumber != "" {
fmt.Printf("\nApplying secrets for preview PR #%s\n", secretsApplyPRNumber)
env, err := config.SynthesizePreviewEnvironment(cfg, secretsApplyPRNumber)
if err != nil {
return fmt.Errorf("synthesizing preview environment: %w", err)
}
return applySecretsForEnv(cfg, &env)
}
if len(secretsApplyEnvs) == 0 {
return fmt.Errorf("specify --env or --pr-number")
}
for _, envKey := range secretsApplyEnvs { for _, envKey := range secretsApplyEnvs {
fmt.Printf("\nApplying secrets for environment: %s\n", envKey) fmt.Printf("\nApplying secrets for environment: %s\n", envKey)
if err := applySecretsForEnv(cfg, envKey); err != nil { env, err := config.ResolveEnvironment(cfg, envKey)
if err != nil {
return fmt.Errorf("env %q: %w", envKey, err)
}
if err := applySecretsForEnv(cfg, &env); err != nil {
return fmt.Errorf("env %q: %w", envKey, err) return fmt.Errorf("env %q: %w", envKey, err)
} }
} }
return nil return nil
} }
func applySecretsForEnv(cfg *config.KforgeConfig, envKey string) error { func applySecretsForEnv(cfg *config.KforgeConfig, env *config.ResolvedEnvironment) error {
env, err := config.ResolveEnvironment(cfg, envKey)
if err != nil {
return err
}
// Basic auth htpasswd secret
if env.Ingress.Auth.Enabled { if env.Ingress.Auth.Enabled {
if err := applyBasicAuthSecret(&env); err != nil { if err := applyBasicAuthSecret(env); err != nil {
return fmt.Errorf("basic auth: %w", err) return fmt.Errorf("basic auth: %w", err)
} }
} }
infra := env.Infrastructure infra := env.Infrastructure
// Database: create credentials Secret before the db-init Job runs.
// Password is alphanumeric-only so it's safe in the Job's shell commands.
if infra.Database != nil {
pgUser := interpolate.PGIdentifier(env.FullName)
if err := applyGenericSecret(
env.FullName+"-db-credentials",
env.Namespace,
map[string]string{
"username": pgUser,
"password": generateAlphanumeric(32),
},
); err != nil {
return fmt.Errorf("db credentials: %w", err)
}
}
if infra.Cache != nil { if infra.Cache != nil {
if err := applyGenericSecret( if err := applyGenericSecret(
env.FullName+"-cache-credentials", env.FullName+"-cache-credentials",
@@ -143,8 +175,6 @@ func applyBasicAuthSecret(env *config.ResolvedEnvironment) error {
for _, username := range auth.Users { for _, username := range auth.Users {
password := generatePassword(32) password := generatePassword(32)
// Generate bcrypt hash using htpasswd (available on most systems)
// or fall back to a simple SHA1 if htpasswd isn't available.
hash, err := generateHTPasswdEntry(username, password) hash, err := generateHTPasswdEntry(username, password)
if err != nil { if err != nil {
return fmt.Errorf("hashing password for %s: %w", username, err) return fmt.Errorf("hashing password for %s: %w", username, err)
@@ -164,32 +194,25 @@ func applyBasicAuthSecret(env *config.ResolvedEnvironment) error {
} }
// generateHTPasswdEntry produces a username:bcrypt_hash string. // generateHTPasswdEntry produces a username:bcrypt_hash string.
// Uses htpasswd binary if available, otherwise uses openssl.
func generateHTPasswdEntry(username, password string) (string, error) { func generateHTPasswdEntry(username, password string) (string, error) {
// Try htpasswd first (apache2-utils package).
if path, err := exec.LookPath("htpasswd"); err == nil { if path, err := exec.LookPath("htpasswd"); err == nil {
out, err := exec.Command(path, "-nbB", username, password).Output() out, err := exec.Command(path, "-nbB", username, password).Output()
if err == nil { if err == nil {
return strings.TrimSpace(string(out)), nil return strings.TrimSpace(string(out)), nil
} }
} }
// Fall back to openssl passwd -apr1 (MD5 crypt, still widely supported).
if path, err := exec.LookPath("openssl"); err == nil { if path, err := exec.LookPath("openssl"); err == nil {
out, err := exec.Command(path, "passwd", "-apr1", password).Output() out, err := exec.Command(path, "passwd", "-apr1", password).Output()
if err == nil { if err == nil {
return username + ":" + strings.TrimSpace(string(out)), nil return username + ":" + strings.TrimSpace(string(out)), nil
} }
} }
return "", fmt.Errorf("neither htpasswd nor openssl found; install apache2-utils") return "", fmt.Errorf("neither htpasswd nor openssl found; install apache2-utils")
} }
// applyGenericSecret creates or updates a Kubernetes Secret using // applyGenericSecret creates or updates a Kubernetes Secret using kubectl.
// kubectl. Skips creation if the secret already exists and --force // Skips creation if the secret already exists and --force was not passed.
// was not passed.
func applyGenericSecret(name, namespace string, data map[string]string) error { func applyGenericSecret(name, namespace string, data map[string]string) error {
// Check if secret already exists.
checkCmd := kubectlCmd("get", "secret", name, "-n", namespace, "--ignore-not-found") checkCmd := kubectlCmd("get", "secret", name, "-n", namespace, "--ignore-not-found")
out, err := checkCmd.Output() out, err := checkCmd.Output()
if err != nil { if err != nil {
@@ -202,7 +225,6 @@ func applyGenericSecret(name, namespace string, data map[string]string) error {
return nil return nil
} }
// Build kubectl create secret generic args.
args := []string{ args := []string{
"create", "secret", "generic", name, "create", "secret", "generic", name,
"-n", namespace, "-n", namespace,
@@ -214,7 +236,6 @@ func applyGenericSecret(name, namespace string, data map[string]string) error {
args = append(args, fmt.Sprintf("--from-literal=%s=%s", k, v)) args = append(args, fmt.Sprintf("--from-literal=%s=%s", k, v))
} }
// Pipe through kubectl apply to handle create-or-update.
createCmd := kubectlCmd(args...) createCmd := kubectlCmd(args...)
yamlBytes, err := createCmd.Output() yamlBytes, err := createCmd.Output()
if err != nil { if err != nil {
@@ -238,9 +259,8 @@ func applyGenericSecret(name, namespace string, data map[string]string) error {
return nil return nil
} }
// kubectlCmd builds a kubectl invocation using the KUBE_HOST, // kubectlCmd builds a kubectl invocation using KUBE_HOST, KUBE_TOKEN,
// KUBE_TOKEN, and KUBE_CERTIFICATE env vars for auth — the same // and KUBE_CERTIFICATE env vars for auth.
// pattern used in your existing Gitea Actions workflow.
func kubectlCmd(args ...string) *exec.Cmd { func kubectlCmd(args ...string) *exec.Cmd {
base := []string{} base := []string{}
@@ -251,11 +271,8 @@ func kubectlCmd(args ...string) *exec.Cmd {
base = append(base, "--token="+token) base = append(base, "--token="+token)
} }
if cert := os.Getenv("KUBE_CERTIFICATE"); cert != "" { if cert := os.Getenv("KUBE_CERTIFICATE"); cert != "" {
// KUBE_CERTIFICATE is the base64-encoded CA cert.
// Decode it to a temp file or pass inline.
decoded, err := base64.StdEncoding.DecodeString(cert) decoded, err := base64.StdEncoding.DecodeString(cert)
if err == nil { if err == nil {
// Write to a temp file for kubectl.
f, err := os.CreateTemp("", "kforge-ca-*.crt") f, err := os.CreateTemp("", "kforge-ca-*.crt")
if err == nil { if err == nil {
_, _ = f.Write(decoded) _, _ = f.Write(decoded)
@@ -264,7 +281,6 @@ func kubectlCmd(args ...string) *exec.Cmd {
} }
} }
} else { } else {
// No cert provided — use insecure skip (matches your current workflow).
base = append(base, "--insecure-skip-tls-verify=true") base = append(base, "--insecure-skip-tls-verify=true")
} }
@@ -276,9 +292,6 @@ func kubectlCmd(args ...string) *exec.Cmd {
// Password generation // Password generation
// ------------------------------------------------------------ // ------------------------------------------------------------
// generatePassword produces a cryptographically random password
// of length n using the full printable ASCII character set.
// Mirrors: tr -dc 'A-Za-z0-9!"#$%&...' </dev/urandom | head -c 32
func generatePassword(n int) string { func generatePassword(n int) string {
b := make([]byte, n) b := make([]byte, n)
for i := range b { for i := range b {
@@ -291,8 +304,6 @@ func generatePassword(n int) string {
return string(b) return string(b)
} }
// generateAlphanumeric produces a random alphanumeric string
// suitable for access keys and usernames.
func generateAlphanumeric(n int) string { func generateAlphanumeric(n int) string {
const chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789" const chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
b := make([]byte, n) b := make([]byte, n)
+3 -26
View File
@@ -141,11 +141,9 @@ type requiredSecret struct {
// kforge.yml requires, based on what's enabled. // kforge.yml requires, based on what's enabled.
func buildRequiredSecrets(cfg *config.KforgeConfig) []requiredSecret { func buildRequiredSecrets(cfg *config.KforgeConfig) []requiredSecret {
secrets := []requiredSecret{ secrets := []requiredSecret{
// Always required — org level // kforge only needs these — DOCKER_USERNAME/PASSWORD are used by
{Name: "DOCKER_USERNAME", Location: "Gitea org secret", Required: true}, // docker/login-action and docker/build-push-action, not by kforge itself.
{Name: "DOCKER_PASSWORD", Location: "Gitea org secret", Required: true}, {Name: "KFORGE_NODE_IP", Location: "Gitea org secret", Description: "Node IP written as the external-dns annotation target"},
{Name: "SOPS_AGE_KEY", Location: "Gitea org secret", Description: "Decrypts .kforge/secrets.enc.yml", Required: true},
{Name: "KFORGE_NODE_IP", Location: "Gitea org secret", Description: "MicroK8s node IP for DNS A records"},
// Always required — repo level // Always required — repo level
{Name: "KUBE_HOST", Location: "Gitea repo secret", Required: true}, {Name: "KUBE_HOST", Location: "Gitea repo secret", Required: true},
@@ -153,27 +151,6 @@ func buildRequiredSecrets(cfg *config.KforgeConfig) []requiredSecret {
{Name: "KUBE_CERTIFICATE", Location: "Gitea repo secret"}, {Name: "KUBE_CERTIFICATE", Location: "Gitea repo secret"},
} }
// DNS secrets
if cfg.DNS.Provider != "" && !cfg.DNS.SkipDNS {
secrets = append(secrets, requiredSecret{
Name: "CLOUDFLARE_API_TOKEN",
Location: "Gitea org secret",
Description: "Zone:Read + DNS:Edit permissions",
Required: true,
})
for _, zone := range cfg.DNS.Cloudflare.Zones {
varName := "CF_ZONE_ID_" + strings.ToUpper(
strings.NewReplacer(".", "_", "-", "_").Replace(zone.Name),
)
secrets = append(secrets, requiredSecret{
Name: varName,
Location: "Gitea org secret",
Description: "Zone ID for " + zone.Name,
Required: true,
})
}
}
// Per-environment infrastructure secrets // Per-environment infrastructure secrets
envKeys := config.EnvironmentKeys(cfg) envKeys := config.EnvironmentKeys(cfg)
sort.Strings(envKeys) sort.Strings(envKeys)
+76 -132
View File
@@ -1,146 +1,90 @@
#!/bin/sh #!/bin/sh
set -e set -e
# INPUT_FILE="$1" COMMAND="${INPUT_COMMAND:-deploy}"
# OUTPUT_FILE="$2" CONFIG="${INPUT_CONFIG:-kforge.yml}"
# AUTO_DEPLOY="$3"
# ---------------------------------------------------------------- # Build a kubeconfig from the standard KUBE_* CI secrets.
# Registry login setup_kube() {
# ---------------------------------------------------------------- [ -z "$KUBE_HOST" ] && return
if [ -n "$INPUT_REGISTRY_USERNAME" ] && [ -n "$INPUT_REGISTRY_PASSWORD" ]; then mkdir -p ~/.kube
echo "Logging in to $INPUT_REGISTRY..." cat > ~/.kube/config <<KUBEEOF
echo "$INPUT_REGISTRY_PASSWORD" | docker login "$INPUT_REGISTRY" \ apiVersion: v1
-u "$INPUT_REGISTRY_USERNAME" --password-stdin kind: Config
fi clusters:
- cluster:
# ---------------------------------------------------------------- certificate-authority-data: ${KUBE_CERTIFICATE}
# Build and push image server: ${KUBE_HOST}
# ---------------------------------------------------------------- name: kforge
cleanup_old_tags() { contexts:
IMAGE="$1" - context:
KEEP="$2" cluster: kforge
user: kforge
echo "Fetching tags for $IMAGE..." name: kforge
current-context: kforge
TAGS=$(curl -s -u "$INPUT_REGISTRY_USERNAME:$INPUT_REGISTRY_PASSWORD" \ users:
"https://$INPUT_REGISTRY/v2/$IMAGE/tags/list" \ - name: kforge
| tr ',' '\n' \ user:
| grep -o '"[a-f0-9]\{7\}"' \ token: ${KUBE_TOKEN}
| tr -d '"') KUBEEOF
chmod 600 ~/.kube/config
COUNT=$(echo "$TAGS" | grep -c .)
DELETE_COUNT=$((COUNT - KEEP))
if [ "$DELETE_COUNT" -le 0 ]; then
echo "Only $COUNT hash tags found, no cleanup needed."
return
fi
echo "Found $COUNT hash tags, deleting oldest $DELETE_COUNT..."
echo "$TAGS" | head -n "$DELETE_COUNT" | while read -r TAG; do
echo "Deleting tag: $TAG..."
DIGEST=$(curl -s -I \
-u "$INPUT_REGISTRY_USERNAME:$INPUT_REGISTRY_PASSWORD" \
-H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
"https://$INPUT_REGISTRY/v2/$IMAGE/manifests/$TAG" \
| grep -i "docker-content-digest" \
| tr -d '\r' \
| awk '{print $2}')
if [ -n "$DIGEST" ]; then
curl -s -X DELETE \
-u "$INPUT_REGISTRY_USERNAME:$INPUT_REGISTRY_PASSWORD" \
"https://$INPUT_REGISTRY/v2/$IMAGE/manifests/$DIGEST"
echo "Deleted $TAG ($DIGEST)"
else
echo "Could not find digest for $TAG, skipping."
fi
done
} }
if [ -n "$INPUT_IMAGE_NAME" ]; then cd "${GITHUB_WORKSPACE:-/github/workspace}"
FULL_IMAGE="$INPUT_REGISTRY/$INPUT_IMAGE_NAME"
if [ -n "$INPUT_IMAGE_TAG" ]; then case "$COMMAND" in
echo "Building image $FULL_IMAGE:$INPUT_IMAGE_TAG..." deploy)
docker build -t "$FULL_IMAGE:$INPUT_IMAGE_TAG" -f "$INPUT_DOCKERFILE" . export KFORGE_IMAGE_TAG="$(git rev-parse --short HEAD)"
setup_kube
echo "Scanning image for vulnerabilities..." kforge validate -c "$CONFIG"
trivy image \ if [ -n "$INPUT_ENV" ]; then
--exit-code 1 \ kforge secrets apply --env "$INPUT_ENV" -c "$CONFIG"
--severity "$INPUT_SCAN_SEVERITY" \ kforge generate --env "$INPUT_ENV" --output .kforge-out -c "$CONFIG"
--no-progress \
"$FULL_IMAGE:$INPUT_IMAGE_TAG"
echo "Scan passed, pushing image..."
docker push "$FULL_IMAGE:$INPUT_IMAGE_TAG"
else else
SHA=$(echo "$GITHUB_SHA" | cut -c1-7) kforge generate --output .kforge-out -c "$CONFIG"
echo "Building image $FULL_IMAGE:latest and $FULL_IMAGE:$SHA..."
docker build \
-t "$FULL_IMAGE:latest" \
-t "$FULL_IMAGE:$SHA" \
-f "$INPUT_DOCKERFILE" .
echo "Scanning image for vulnerabilities..."
trivy image \
--exit-code 1 \
--severity "$INPUT_SCAN_SEVERITY" \
--no-progress \
"$FULL_IMAGE:latest"
echo "Scan passed, pushing image..."
docker push "$FULL_IMAGE:latest"
docker push "$FULL_IMAGE:$SHA"
cleanup_old_tags "$INPUT_IMAGE_NAME" "${INPUT_MAX_TAGS:-5}"
fi fi
fi kubectl apply -f .kforge-out/ --insecure-skip-tls-verify --validate=false
NAMESPACE="${INPUT_NAMESPACE:-${INPUT_ENV}}"
if [ -n "$NAMESPACE" ]; then
kubectl rollout restart deployment -n "$NAMESPACE" --insecure-skip-tls-verify || true
fi
;;
# ---------------------------------------------------------------- preview-up)
# Generate Kubernetes YAML [ -z "$INPUT_PR_NUMBER" ] && echo "::error::pr_number input is required for preview-up" && exit 1
# ---------------------------------------------------------------- setup_kube
echo "Generating Kubernetes YAML from .kforge.yml" kforge secrets apply --pr-number "$INPUT_PR_NUMBER" -c "$CONFIG"
/usr/local/bin/kforge generate kforge generate --pr-number "$INPUT_PR_NUMBER" --output .kforge-out -c "$CONFIG"
kubectl apply -f .kforge-out/ --insecure-skip-tls-verify --validate=false
NS="${INPUT_NAMESPACE_PREFIX:-preview-pr}-${INPUT_PR_NUMBER}"
kubectl rollout status deployment -n "$NS" --timeout=120s --insecure-skip-tls-verify || true
;;
# ---------------------------------------------------------------- preview-down)
# Deploy to Kubernetes [ -z "$INPUT_PR_NUMBER" ] && echo "::error::pr_number input is required for preview-down" && exit 1
# ---------------------------------------------------------------- setup_kube
# Build kubeconfig from token-based credentials NS="${INPUT_NAMESPACE_PREFIX:-preview-pr}-${INPUT_PR_NUMBER}"
echo "Configuring kubectl..." kubectl delete namespace "$NS" --ignore-not-found --insecure-skip-tls-verify
;;
# Try writing the cert and check if it worked validate)
echo "$INPUT_KUBE_CERTIFICATE" | base64 -d > /tmp/kube-ca.crt 2>&1 kforge validate -c "$CONFIG"
echo "Cert file size: $(wc -c < /tmp/kube-ca.crt)" ;;
echo "Cert file contents: $(cat /tmp/kube-ca.crt | head -1)"
kubectl config set-cluster default \ secrets)
--server="$INPUT_KUBE_HOST" \ setup_kube
--certificate-authority=/tmp/kube-ca.crt if [ -n "$INPUT_PR_NUMBER" ]; then
kforge secrets apply --pr-number "$INPUT_PR_NUMBER" -c "$CONFIG"
elif [ -n "$INPUT_ENV" ]; then
kforge secrets apply --env "$INPUT_ENV" -c "$CONFIG"
else
echo "::error::env or pr_number input is required for the secrets command"
exit 1
fi
;;
kubectl config set-credentials default \ *)
--token="$INPUT_KUBE_TOKEN" echo "::error::Unknown command '$COMMAND'. Valid: deploy, preview-up, preview-down, validate, secrets"
exit 1
kubectl config set-context default \ ;;
--cluster=default \ esac
--user=default
kubectl config use-context default
# Create/update regcred secret idempotently
# echo "Creating regcred secret..."
# kubectl create secret docker-registry regcred \
# --docker-server="$INPUT_REGISTRY" \
# --docker-username="$INPUT_REGISTRY_USERNAME" \
# --docker-password="$INPUT_REGISTRY_PASSWORD" \
# --dry-run=client -o yaml | kubectl apply -f - --insecure-skip-tls-verify --validate=false
echo "Deploying to Kubernetes..."
kubectl apply --insecure-skip-tls-verify --validate=false -f ./.kforge-out/
echo "Deploy complete."
echo "Cleanup"
rm -f /tmp/kube-ca.crt
+14 -10
View File
@@ -6,23 +6,19 @@ registry:
url: registry.natelubitz.com url: registry.natelubitz.com
pull_secret: regcred pull_secret: regcred
# DNS records are managed by external-dns inside the cluster.
# Set target to your node's public IP or a static hostname.
# ${KFORGE_NODE_IP} is resolved from the CI secret at generate time.
dns: dns:
provider: cloudflare target: ${KFORGE_NODE_IP}
cloudflare:
api_token: ${CLOUDFLARE_API_TOKEN}
zones:
- name: natelubitz.com
zone_id: ${CF_ZONE_ID_NATELUBITZ}
- name: midtermtenant.com
zone_id: ${CF_ZONE_ID_MIDTERM}
proxied: false
node_ip: ${KFORGE_NODE_IP}
cluster: cluster:
tls_issuer: letsencrypt-prod tls_issuer: letsencrypt-prod
ingress_class: nginx ingress_class: nginx
cnpg: cnpg:
host: cnpg-main-rw.default.svc.cluster.local host: cnpg-main-rw.default.svc.cluster.local
cluster_name: cnpg-main
superuser_secret: cnpg-main-superuser
defaults: defaults:
port: 3000 port: 3000
@@ -36,6 +32,14 @@ infrastructure:
provider: valkey provider: valkey
mode: standalone mode: standalone
# Preview environments are deployed per pull request.
# Run `kforge gitea-preview` to generate .gitea/workflows/preview.yml
preview:
enabled: true
base_environment: staging
namespace_prefix: preview-pr
hostname_template: "pr-${PR_NUMBER}.${name}.natelubitz.com"
environments: environments:
staging: staging:
namespace: staging namespace: staging
+1 -1
View File
@@ -4,7 +4,7 @@ go 1.22
require ( require (
github.com/spf13/cobra v1.8.0 github.com/spf13/cobra v1.8.0
github.com/spf13/pflag v1.0.10 github.com/spf13/pflag v1.0.5
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
) )
-2
View File
@@ -6,8 +6,6 @@ github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho= github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+105 -24
View File
@@ -1,5 +1,13 @@
package config package config
import (
"fmt"
"os"
"strings"
"kforge/pkg/interpolate"
)
// ------------------------------------------------------------ // ------------------------------------------------------------
// Default values — single source of truth for every default // Default values — single source of truth for every default
// referenced in the schema. Change a default here and it // referenced in the schema. Change a default here and it
@@ -10,6 +18,8 @@ const (
DefaultTLSIssuer = "letsencrypt-prod" DefaultTLSIssuer = "letsencrypt-prod"
DefaultIngressClass = "nginx" DefaultIngressClass = "nginx"
DefaultCNPGHost = "cnpg-main-rw.default.svc.cluster.local" DefaultCNPGHost = "cnpg-main-rw.default.svc.cluster.local"
DefaultCNPGClusterName = "cnpg-main"
DefaultCNPGSuperuserSecret = "cnpg-main-superuser"
DefaultNamespacePattern = "${env}" DefaultNamespacePattern = "${env}"
DefaultImagePullPolicy = "Always" DefaultImagePullPolicy = "Always"
DefaultServiceType = "ClusterIP" DefaultServiceType = "ClusterIP"
@@ -40,6 +50,8 @@ const (
DefaultRestartPolicy = "OnFailure" DefaultRestartPolicy = "OnFailure"
DefaultConcurrencyPolicy = "Forbid" DefaultConcurrencyPolicy = "Forbid"
DefaultPreviewNamespacePrefix = "preview-pr"
) )
// boolPtr / intPtr are helpers for pointer defaults. // boolPtr / intPtr are helpers for pointer defaults.
@@ -76,6 +88,12 @@ func applyClusterDefaults(c *ClusterConfig) {
if c.CNPG.Host == "" { if c.CNPG.Host == "" {
c.CNPG.Host = DefaultCNPGHost c.CNPG.Host = DefaultCNPGHost
} }
if c.CNPG.ClusterName == "" {
c.CNPG.ClusterName = DefaultCNPGClusterName
}
if c.CNPG.SuperuserSecret == "" {
c.CNPG.SuperuserSecret = DefaultCNPGSuperuserSecret
}
if c.NamespacePattern == "" { if c.NamespacePattern == "" {
c.NamespacePattern = DefaultNamespacePattern c.NamespacePattern = DefaultNamespacePattern
} }
@@ -85,7 +103,8 @@ func applyRegistryDefaults(r *RegistryConfig, m *MetaConfig) {
if r.URL == "" { if r.URL == "" {
r.URL = DefaultRegistryURL r.URL = DefaultRegistryURL
} }
if r.PullSecret == "" { // Insecure (in-cluster) registries need no imagePullSecret.
if r.PullSecret == "" && !r.Insecure {
r.PullSecret = DefaultPullSecret r.PullSecret = DefaultPullSecret
} }
if r.Repository == "" { if r.Repository == "" {
@@ -156,9 +175,7 @@ func applyResourceDefaults(r *ResourceConfig) {
} }
// applyRootInfraDefaults sets provider/mode defaults on the root // applyRootInfraDefaults sets provider/mode defaults on the root
// infrastructure block. The enabled flag is handled by the merge // infrastructure block.
// step: if a block exists at root with no explicit enabled:false,
// it is considered enabled.
func applyRootInfraDefaults(infra *InfrastructureConfig) { func applyRootInfraDefaults(infra *InfrastructureConfig) {
if infra.Cache != nil { if infra.Cache != nil {
if infra.Cache.Provider == "" { if infra.Cache.Provider == "" {
@@ -243,6 +260,12 @@ type ResolvedEnvironment struct {
TLSIssuer string TLSIssuer string
IngressClass string IngressClass string
CNPGHost string CNPGHost string
CNPGClusterName string
CNPGSuperuserSecret string
// DNS (for external-dns Ingress annotations)
DNSTarget string
SkipDNS bool
// Lifecycle // Lifecycle
Lifecycle LifecycleConfig Lifecycle LifecycleConfig
@@ -285,6 +308,11 @@ func ResolveEnvironment(cfg *KforgeConfig, envKey string) (ResolvedEnvironment,
if imageTag == "" { if imageTag == "" {
imageTag = DefaultImageTag imageTag = DefaultImageTag
} }
// Allow the action entrypoint (or CI) to override the image tag at generate
// time without modifying kforge.yml (e.g. KFORGE_IMAGE_TAG=abc1234).
if override := os.Getenv("KFORGE_IMAGE_TAG"); override != "" {
imageTag = override
}
image := registry.URL + "/" + registry.Repository + ":" + imageTag image := registry.URL + "/" + registry.Repository + ":" + imageTag
replicas := *cfg.Defaults.Replicas replicas := *cfg.Defaults.Replicas
@@ -318,6 +346,12 @@ func ResolveEnvironment(cfg *KforgeConfig, envKey string) (ResolvedEnvironment,
Auth: auth, Auth: auth,
} }
// Resolve DNS target token (${KFORGE_NODE_IP} etc.) from env.
tokens := interpolate.FromEnvironment(
cfg.Meta.Name, cfg.Meta.Tenant, envKey, prefix, fullName, imageTag, namespace,
)
dnsTarget := interpolate.Apply(cfg.DNS.Target, tokens)
return ResolvedEnvironment{ return ResolvedEnvironment{
EnvKey: envKey, EnvKey: envKey,
EnvPrefix: prefix, EnvPrefix: prefix,
@@ -338,10 +372,75 @@ func ResolveEnvironment(cfg *KforgeConfig, envKey string) (ResolvedEnvironment,
TLSIssuer: cfg.Cluster.TLSIssuer, TLSIssuer: cfg.Cluster.TLSIssuer,
IngressClass: cfg.Cluster.IngressClass, IngressClass: cfg.Cluster.IngressClass,
CNPGHost: cfg.Cluster.CNPG.Host, CNPGHost: cfg.Cluster.CNPG.Host,
CNPGClusterName: cfg.Cluster.CNPG.ClusterName,
CNPGSuperuserSecret: cfg.Cluster.CNPG.SuperuserSecret,
DNSTarget: dnsTarget,
SkipDNS: cfg.DNS.SkipDNS,
Lifecycle: lifecycle, Lifecycle: lifecycle,
}, nil }, nil
} }
// ------------------------------------------------------------
// SynthesizePreviewEnvironment creates a ResolvedEnvironment
// for a PR preview from the preview config + a base environment.
// ------------------------------------------------------------
// SynthesizePreviewEnvironment derives a preview environment by
// cloning the base environment and overriding namespace, hostname,
// full name, and image tag for the given PR number.
func SynthesizePreviewEnvironment(cfg *KforgeConfig, prNumber string) (ResolvedEnvironment, error) {
if !cfg.Preview.Enabled {
return ResolvedEnvironment{}, fmt.Errorf("preview is not enabled in kforge.yml")
}
if cfg.Preview.BaseEnvironment == "" {
return ResolvedEnvironment{}, fmt.Errorf("preview.base_environment is required")
}
env, err := ResolveEnvironment(cfg, cfg.Preview.BaseEnvironment)
if err != nil {
return ResolvedEnvironment{}, fmt.Errorf("resolving base environment %q: %w", cfg.Preview.BaseEnvironment, err)
}
nsPrefix := cfg.Preview.NamespacePrefix
if nsPrefix == "" {
nsPrefix = DefaultPreviewNamespacePrefix
}
env.EnvKey = "preview-" + prNumber
env.Namespace = nsPrefix + "-" + prNumber
env.FullName = "pr" + prNumber + "-" + cfg.Meta.Tenant + "-" + cfg.Meta.Name
// Override image tag to a PR-specific tag.
if idx := strings.LastIndex(env.Image, ":"); idx >= 0 {
env.Image = env.Image[:idx+1] + "pr-" + prNumber
}
// Override ingress hostname using the template.
hostnameTemplate := cfg.Preview.HostnameTemplate
if hostnameTemplate == "" {
hostnameTemplate = "pr-${PR_NUMBER}.${name}.example.com"
}
previewTokens := interpolate.Tokens{
"PR_NUMBER": prNumber,
"name": cfg.Meta.Name,
"tenant": cfg.Meta.Tenant,
}
hostname := interpolate.Apply(hostnameTemplate, previewTokens)
env.Ingress = IngressConfig{
Hosts: []IngressHost{{
Hostname: hostname,
TLS: true,
DNSRecord: true,
}},
Auth: IngressAuth{
SecretName: env.FullName + "-basic-auth",
},
}
return env, nil
}
// ------------------------------------------------------------ // ------------------------------------------------------------
// Internal resolution helpers // Internal resolution helpers
// ------------------------------------------------------------ // ------------------------------------------------------------
@@ -350,7 +449,6 @@ func resolveEnvPrefix(envKey string, override *string) string {
if override != nil && *override != "" { if override != nil && *override != "" {
return *override return *override
} }
// Default: first 4 chars of envKey, or full key if shorter.
if len(envKey) <= 4 { if len(envKey) <= 4 {
return envKey return envKey
} }
@@ -361,13 +459,10 @@ func resolveNamespace(envKey, explicit, pattern string) string {
if explicit != "" { if explicit != "" {
return explicit return explicit
} }
// Apply the namespace pattern (simple token replace here; if pattern == "" {
// full interpolation runs later via the interpolate package).
result := pattern
if result == "" {
return envKey return envKey
} }
return result return pattern
} }
func resolveFullName(cfg *KforgeConfig, prefix string, env EnvironmentConfig) string { func resolveFullName(cfg *KforgeConfig, prefix string, env EnvironmentConfig) string {
@@ -379,7 +474,6 @@ func resolveFullName(cfg *KforgeConfig, prefix string, env EnvironmentConfig) st
func resolveRegistry(cfg *KforgeConfig, env EnvironmentConfig) RegistryConfig { func resolveRegistry(cfg *KforgeConfig, env EnvironmentConfig) RegistryConfig {
base := cfg.Registry base := cfg.Registry
// Resolve the repository token now that we have meta values.
if base.Repository == "${tenant}/${name}" || base.Repository == "" { if base.Repository == "${tenant}/${name}" || base.Repository == "" {
base.Repository = cfg.Meta.Tenant + "/" + cfg.Meta.Name base.Repository = cfg.Meta.Tenant + "/" + cfg.Meta.Name
} }
@@ -428,15 +522,6 @@ func mergeEnvVars(base, override []EnvVarConfig) []EnvVarConfig {
// mergeInfrastructure performs a shallow merge of root infra // mergeInfrastructure performs a shallow merge of root infra
// defaults with per-environment overrides. // defaults with per-environment overrides.
//
// Rules:
// - If root has a service block with no explicit enabled:false,
// it is enabled in every environment.
// - An env block with enabled:false disables the service.
// - An env block with partial fields overrides only those fields;
// everything else inherits from root.
// - If root has no block for a service, env can still enable it
// by providing its own block (enabled defaults to true if present).
func mergeInfrastructure(root, env *InfrastructureConfig) ResolvedInfrastructure { func mergeInfrastructure(root, env *InfrastructureConfig) ResolvedInfrastructure {
return ResolvedInfrastructure{ return ResolvedInfrastructure{
Database: mergeDatabase(root.Database, env.Database), Database: mergeDatabase(root.Database, env.Database),
@@ -635,14 +720,12 @@ func resolveCronJobs(jobs []CronJobConfig, deploymentImage string, deploymentEnv
for _, job := range jobs { for _, job := range jobs {
r := ResolvedCronJob{CronJobConfig: job} r := ResolvedCronJob{CronJobConfig: job}
// Image
if job.ImageOverride != nil && *job.ImageOverride != "" { if job.ImageOverride != nil && *job.ImageOverride != "" {
r.Image = *job.ImageOverride r.Image = *job.ImageOverride
} else { } else {
r.Image = deploymentImage r.Image = deploymentImage
} }
// Env vars: deployment vars first, then job-specific (with merge)
inheritEnv := job.InheritEnv == nil || *job.InheritEnv inheritEnv := job.InheritEnv == nil || *job.InheritEnv
if inheritEnv { if inheritEnv {
r.EnvVars = mergeEnvVars(deploymentEnvVars, job.EnvVars) r.EnvVars = mergeEnvVars(deploymentEnvVars, job.EnvVars)
@@ -650,12 +733,10 @@ func resolveCronJobs(jobs []CronJobConfig, deploymentImage string, deploymentEnv
r.EnvVars = job.EnvVars r.EnvVars = job.EnvVars
} }
// Resources
if job.Resources == nil { if job.Resources == nil {
r.Resources = &defaultResources r.Resources = &defaultResources
} }
// Tuning defaults
if r.RestartPolicy == "" { if r.RestartPolicy == "" {
r.RestartPolicy = DefaultRestartPolicy r.RestartPolicy = DefaultRestartPolicy
} }
+4
View File
@@ -3,6 +3,7 @@ package config
import ( import (
"fmt" "fmt"
"os" "os"
"strings"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
) )
@@ -39,6 +40,9 @@ func validate(cfg *KforgeConfig) error {
if cfg.Meta.Tenant == "" { if cfg.Meta.Tenant == "" {
return fmt.Errorf("meta.tenant is required") return fmt.Errorf("meta.tenant is required")
} }
if cfg.ActionRef != "" && (strings.Contains(cfg.ActionRef, "@") && !strings.Contains(cfg.ActionRef, "@sha256:")) {
return fmt.Errorf("action_ref %q looks like a git ref — use a Docker image tag instead (e.g. registry.example.com/infra/kforge:latest)", cfg.ActionRef)
}
if len(cfg.Environments) == 0 { if len(cfg.Environments) == 0 {
return fmt.Errorf("at least one environment must be defined") return fmt.Errorf("at least one environment must be defined")
} }
+54 -23
View File
@@ -12,7 +12,15 @@ type KforgeConfig struct {
Cluster ClusterConfig `yaml:"cluster"` Cluster ClusterConfig `yaml:"cluster"`
Defaults DefaultsConfig `yaml:"defaults"` Defaults DefaultsConfig `yaml:"defaults"`
Infrastructure InfrastructureConfig `yaml:"infrastructure"` Infrastructure InfrastructureConfig `yaml:"infrastructure"`
Preview PreviewConfig `yaml:"preview,omitempty"`
Environments map[string]EnvironmentConfig `yaml:"environments"` Environments map[string]EnvironmentConfig `yaml:"environments"`
// ActionRef is the Docker image reference for the pre-built kforge action
// (e.g. "registry.example.com/infra/kforge:latest"). When set, kforge
// gitea-actions and kforge gitea-preview generate `uses: docker://<ref>`
// steps instead of installing and running kforge inline. The kforge repo
// must publish this image on each release.
ActionRef string `yaml:"action_ref,omitempty"`
} }
// ------------------------------------------------------------ // ------------------------------------------------------------
@@ -33,29 +41,31 @@ type MetaConfig struct {
type RegistryConfig struct { type RegistryConfig struct {
URL string `yaml:"url"` URL string `yaml:"url"`
Repository string `yaml:"repository,omitempty"` // default: ${tenant}/${name} Repository string `yaml:"repository,omitempty"` // default: ${tenant}/${name}
PullSecret string `yaml:"pull_secret,omitempty"` // default: regcred PullSecret string `yaml:"pull_secret,omitempty"` // default: regcred; set to "" to disable
// Insecure marks the registry as HTTP-only (no TLS). Skips docker login,
// omits imagePullSecrets from manifests, and configures buildkitd for
// plain-HTTP pushes. Typical for in-cluster registries accessed via
// ClusterIP/service DNS rather than an Ingress.
Insecure bool `yaml:"insecure,omitempty"`
} }
// ------------------------------------------------------------ // ------------------------------------------------------------
// DNS // DNS
//
// DNS records and TLS are managed inside the cluster:
// - external-dns reads Ingress annotations and creates DNS records
// - cert-manager issues TLS certificates via a ClusterIssuer
//
// kforge emits the appropriate annotations on the Ingress resource
// for each host with dns_record: true.
// ------------------------------------------------------------ // ------------------------------------------------------------
type DNSConfig struct { type DNSConfig struct {
Provider string `yaml:"provider"` // "cloudflare" // Target is the value written to the external-dns target annotation.
Cloudflare CloudflareConfig `yaml:"cloudflare"` // Typically the cluster node's public IP or a static hostname.
NodeIP string `yaml:"node_ip,omitempty"` // default: ${KFORGE_NODE_IP} // Supports token interpolation (e.g. ${KFORGE_NODE_IP}).
SkipDNS bool `yaml:"skip_dns,omitempty"` Target string `yaml:"target,omitempty"`
} SkipDNS bool `yaml:"skip_dns,omitempty"` // true = no external-dns annotations
type CloudflareConfig struct {
APIToken string `yaml:"api_token"`
Zones []ZoneEntry `yaml:"zones"`
Proxied bool `yaml:"proxied,omitempty"`
}
type ZoneEntry struct {
Name string `yaml:"name"`
ZoneID string `yaml:"zone_id"`
} }
// ------------------------------------------------------------ // ------------------------------------------------------------
@@ -71,7 +81,8 @@ type ClusterConfig struct {
type CNPGConfig struct { type CNPGConfig struct {
Host string `yaml:"host,omitempty"` // default: cnpg-main-rw.default.svc.cluster.local Host string `yaml:"host,omitempty"` // default: cnpg-main-rw.default.svc.cluster.local
HostOverride *string `yaml:"host_override,omitempty"` ClusterName string `yaml:"cluster_name,omitempty"` // default: cnpg-main
SuperuserSecret string `yaml:"superuser_secret,omitempty"` // default: cnpg-main-superuser
} }
// ------------------------------------------------------------ // ------------------------------------------------------------
@@ -155,9 +166,6 @@ type InfrastructureConfig struct {
// InfraBase holds the common enabled flag present on every // InfraBase holds the common enabled flag present on every
// infrastructure service. Embedded in each service config. // infrastructure service. Embedded in each service config.
type InfraBase struct { type InfraBase struct {
// Enabled defaults to true if the block exists in the root
// infrastructure section, and inherits that value in envs.
// Set explicitly to false to disable for a specific env.
Enabled *bool `yaml:"enabled,omitempty"` Enabled *bool `yaml:"enabled,omitempty"`
} }
@@ -197,6 +205,31 @@ type MonitoringInfraConfig struct {
MetricsPort *int `yaml:"metrics_port,omitempty"` // default: defaults.port MetricsPort *int `yaml:"metrics_port,omitempty"` // default: defaults.port
} }
// ------------------------------------------------------------
// Preview environments
// ------------------------------------------------------------
// PreviewConfig defines how PR preview environments are deployed.
// When enabled, `kforge gitea-preview` generates a workflow that
// deploys an isolated environment per pull request.
type PreviewConfig struct {
// Enabled must be true for kforge gitea-preview to generate a workflow.
Enabled bool `yaml:"enabled,omitempty"`
// BaseEnvironment is the environment whose infrastructure and env var
// settings are inherited (e.g. "staging"). Required when enabled.
BaseEnvironment string `yaml:"base_environment,omitempty"`
// NamespacePrefix is prepended to the PR number to form the namespace.
// Default: "preview-pr" → namespace "preview-pr-123".
NamespacePrefix string `yaml:"namespace_prefix,omitempty"`
// HostnameTemplate is the ingress hostname for the preview, using
// ${PR_NUMBER}, ${name}, and ${tenant} as substitution tokens.
// Example: "pr-${PR_NUMBER}.${name}.example.com"
HostnameTemplate string `yaml:"hostname_template,omitempty"`
}
// ------------------------------------------------------------ // ------------------------------------------------------------
// Environment // Environment
// ------------------------------------------------------------ // ------------------------------------------------------------
@@ -233,7 +266,7 @@ type IngressConfig struct {
type IngressHost struct { type IngressHost struct {
Hostname string `yaml:"hostname"` Hostname string `yaml:"hostname"`
TLS bool `yaml:"tls"` TLS bool `yaml:"tls"`
DNSRecord bool `yaml:"dns_record,omitempty"` DNSRecord bool `yaml:"dns_record,omitempty"` // emit external-dns annotations
} }
type IngressAuth struct { type IngressAuth struct {
@@ -267,8 +300,6 @@ type CronJobConfig struct {
// ------------------------------------------------------------ // ------------------------------------------------------------
type LifecycleConfig struct { type LifecycleConfig struct {
// If false (default), kforge renames resources when meta.name changes.
// If true, kforge deletes old resources after delete_grace_seconds.
Delete bool `yaml:"delete,omitempty"` Delete bool `yaml:"delete,omitempty"`
DeleteGraceSeconds int `yaml:"delete_grace_seconds,omitempty"` // default: 300 DeleteGraceSeconds int `yaml:"delete_grace_seconds,omitempty"` // default: 300
} }
-301
View File
@@ -1,301 +0,0 @@
// Package dns provides a provider-agnostic interface for DNS
// record management, with a Cloudflare implementation.
//
// Adding a new provider (Route53, Porkbun, etc.):
// 1. Implement the Provider interface below.
// 2. Add a case in NewProvider().
// 3. The rest of kforge uses Provider — no other changes needed.
package dns
import (
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
"kforge/internal/config"
)
// ------------------------------------------------------------
// Provider interface
// ------------------------------------------------------------
// Provider is the DNS provider contract. Implementations must
// be idempotent — calling EnsureARecord twice with the same
// inputs must not error or create duplicates.
type Provider interface {
// EnsureARecord creates an A record for hostname pointing to
// ip if one does not already exist. If a record exists with a
// different IP, it is updated. No-ops if already correct.
EnsureARecord(hostname, ip string) error
// DeleteARecord removes the A record for hostname if it exists.
// No-ops if it does not exist.
DeleteARecord(hostname string) error
}
// NewProvider returns the configured DNS provider.
func NewProvider(cfg config.DNSConfig) (Provider, error) {
switch cfg.Provider {
case "cloudflare":
return newCloudflareProvider(cfg)
case "":
return &noopProvider{}, nil
default:
return nil, fmt.Errorf("unknown dns provider %q (supported: cloudflare)", cfg.Provider)
}
}
// noopProvider satisfies the interface when DNS management is
// disabled (skip_dns: true or no provider configured).
type noopProvider struct{}
func (n *noopProvider) EnsureARecord(hostname, ip string) error { return nil }
func (n *noopProvider) DeleteARecord(hostname string) error { return nil }
// ------------------------------------------------------------
// Cloudflare implementation
// ------------------------------------------------------------
const cfAPIBase = "https://api.cloudflare.com/client/v4"
type cloudflareProvider struct {
apiToken string
zones []config.ZoneEntry // sorted longest-first for matching
proxied bool
client *http.Client
}
func newCloudflareProvider(cfg config.DNSConfig) (*cloudflareProvider, error) {
if cfg.Cloudflare.APIToken == "" {
return nil, fmt.Errorf("cloudflare.api_token is required")
}
if len(cfg.Cloudflare.Zones) == 0 {
return nil, fmt.Errorf("cloudflare.zones must have at least one entry")
}
return &cloudflareProvider{
apiToken: cfg.Cloudflare.APIToken,
zones: cfg.Cloudflare.Zones,
proxied: cfg.Cloudflare.Proxied,
client: &http.Client{Timeout: 15 * time.Second},
}, nil
}
// zoneForHostname finds the zone whose name is the longest suffix
// of hostname. This handles both "app.example.com" → "example.com"
// and "app.sub.example.co.uk" → "example.co.uk" if that zone exists.
func (c *cloudflareProvider) zoneForHostname(hostname string) (config.ZoneEntry, error) {
var best config.ZoneEntry
bestLen := 0
for _, z := range c.zones {
if strings.HasSuffix(hostname, z.Name) && len(z.Name) > bestLen {
best = z
bestLen = len(z.Name)
}
}
if bestLen == 0 {
return config.ZoneEntry{}, fmt.Errorf("no configured zone matches hostname %q", hostname)
}
return best, nil
}
// EnsureARecord is idempotent: creates if absent, updates if IP
// differs, no-ops if already correct.
func (c *cloudflareProvider) EnsureARecord(hostname, ip string) error {
zone, err := c.zoneForHostname(hostname)
if err != nil {
return err
}
existing, err := c.getRecord(zone.ZoneID, hostname, "A")
if err != nil {
return fmt.Errorf("checking existing record: %w", err)
}
if existing != nil {
if existing.Content == ip {
fmt.Printf(" dns: A record %s → %s already correct, skipping\n", hostname, ip)
return nil
}
fmt.Printf(" dns: updating A record %s → %s (was %s)\n", hostname, ip, existing.Content)
return c.updateRecord(zone.ZoneID, existing.ID, hostname, ip)
}
fmt.Printf(" dns: creating A record %s → %s\n", hostname, ip)
return c.createRecord(zone.ZoneID, hostname, ip)
}
// DeleteARecord removes the A record for hostname if it exists.
func (c *cloudflareProvider) DeleteARecord(hostname string) error {
zone, err := c.zoneForHostname(hostname)
if err != nil {
return err
}
existing, err := c.getRecord(zone.ZoneID, hostname, "A")
if err != nil {
return fmt.Errorf("checking existing record: %w", err)
}
if existing == nil {
fmt.Printf(" dns: A record %s not found, skipping delete\n", hostname)
return nil
}
fmt.Printf(" dns: deleting A record %s\n", hostname)
return c.deleteRecord(zone.ZoneID, existing.ID)
}
// ------------------------------------------------------------
// Cloudflare API helpers
// ------------------------------------------------------------
type cfRecord struct {
ID string `json:"id"`
Name string `json:"name"`
Type string `json:"type"`
Content string `json:"content"`
Proxied bool `json:"proxied"`
TTL int `json:"ttl"`
}
type cfListResponse struct {
Success bool `json:"success"`
Errors []cfError `json:"errors"`
Result []cfRecord `json:"result"`
}
type cfSingleResponse struct {
Success bool `json:"success"`
Errors []cfError `json:"errors"`
Result cfRecord `json:"result"`
}
type cfError struct {
Code int `json:"code"`
Message string `json:"message"`
}
func (e cfError) Error() string {
return fmt.Sprintf("CF %d: %s", e.Code, e.Message)
}
func (c *cloudflareProvider) getRecord(zoneID, name, recType string) (*cfRecord, error) {
url := fmt.Sprintf("%s/zones/%s/dns_records?type=%s&name=%s", cfAPIBase, zoneID, recType, name)
resp, err := c.do("GET", url, nil)
if err != nil {
return nil, err
}
defer resp.Body.Close()
var list cfListResponse
if err := json.NewDecoder(resp.Body).Decode(&list); err != nil {
return nil, fmt.Errorf("decoding response: %w", err)
}
if !list.Success {
return nil, cfErrors(list.Errors)
}
if len(list.Result) == 0 {
return nil, nil
}
return &list.Result[0], nil
}
func (c *cloudflareProvider) createRecord(zoneID, name, ip string) error {
body := fmt.Sprintf(`{"type":"A","name":%q,"content":%q,"ttl":1,"proxied":%v}`,
name, ip, c.proxied)
url := fmt.Sprintf("%s/zones/%s/dns_records", cfAPIBase, zoneID)
resp, err := c.do("POST", url, strings.NewReader(body))
if err != nil {
return err
}
defer resp.Body.Close()
var result cfSingleResponse
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return fmt.Errorf("decoding response: %w", err)
}
if !result.Success {
return cfErrors(result.Errors)
}
return nil
}
func (c *cloudflareProvider) updateRecord(zoneID, recordID, name, ip string) error {
body := fmt.Sprintf(`{"type":"A","name":%q,"content":%q,"ttl":1,"proxied":%v}`,
name, ip, c.proxied)
url := fmt.Sprintf("%s/zones/%s/dns_records/%s", cfAPIBase, zoneID, recordID)
resp, err := c.do("PUT", url, strings.NewReader(body))
if err != nil {
return err
}
defer resp.Body.Close()
var result cfSingleResponse
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return fmt.Errorf("decoding response: %w", err)
}
if !result.Success {
return cfErrors(result.Errors)
}
return nil
}
func (c *cloudflareProvider) deleteRecord(zoneID, recordID string) error {
url := fmt.Sprintf("%s/zones/%s/dns_records/%s", cfAPIBase, zoneID, recordID)
resp, err := c.do("DELETE", url, nil)
if err != nil {
return err
}
defer resp.Body.Close()
// Cloudflare returns {"result":{"id":"..."}} on success — we
// don't need to parse it, just check for HTTP errors.
body, _ := io.ReadAll(resp.Body)
if resp.StatusCode >= 400 {
return fmt.Errorf("delete failed (%d): %s", resp.StatusCode, string(body))
}
return nil
}
func (c *cloudflareProvider) do(method, url string, body io.Reader) (*http.Response, error) {
req, err := http.NewRequest(method, url, body)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+c.apiToken)
req.Header.Set("Content-Type", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("cloudflare API %s %s: %w", method, url, err)
}
return resp, nil
}
func cfErrors(errs []cfError) error {
msgs := make([]string, len(errs))
for i, e := range errs {
msgs[i] = e.Error()
}
return fmt.Errorf("cloudflare API errors: %s", strings.Join(msgs, "; "))
}
// ------------------------------------------------------------
// EnsureRecordsForEnvironment — high-level helper used by
// the apply command and generate pipeline.
// ------------------------------------------------------------
// EnsureRecordsForEnvironment creates A records for all ingress
// hosts in the environment that have dns_record: true.
func EnsureRecordsForEnvironment(provider Provider, env *config.ResolvedEnvironment, nodeIP string) error {
for _, host := range env.Ingress.Hosts {
if !host.DNSRecord {
continue
}
if err := provider.EnsureARecord(host.Hostname, nodeIP); err != nil {
return fmt.Errorf("ensuring A record for %s: %w", host.Hostname, err)
}
}
return nil
}
+248 -61
View File
@@ -23,9 +23,8 @@ type GiteaActionsOptions struct {
// that builds the Docker image and deploys to each environment // that builds the Docker image and deploys to each environment
// using kforge generate + kubectl apply. // using kforge generate + kubectl apply.
// //
// The generated file is designed to replace your existing // DNS is handled by external-dns via Ingress annotations — no
// hand-written workflow. It assumes kforge is available in the // separate DNS step is needed in the workflow.
// PATH (either pre-installed on the runner or fetched as a step).
func GenerateGiteaActions(cfg *config.KforgeConfig, opts GiteaActionsOptions) (string, error) { func GenerateGiteaActions(cfg *config.KforgeConfig, opts GiteaActionsOptions) (string, error) {
if opts.Branch == "" { if opts.Branch == "" {
opts.Branch = "main" opts.Branch = "main"
@@ -39,20 +38,19 @@ func GenerateGiteaActions(cfg *config.KforgeConfig, opts GiteaActionsOptions) (s
var b strings.Builder var b strings.Builder
writeGiteaHeader(&b, cfg, opts) writeDeployHeader(&b, cfg, opts)
writeGiteaJobs(&b, cfg, opts) writeDeployJobs(&b, cfg, opts)
return b.String(), nil return b.String(), nil
} }
func writeGiteaHeader(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) { func writeDeployHeader(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) {
fmt.Fprintf(b, "# Generated by kforge — do not edit manually.\n") fmt.Fprintf(b, "# Generated by kforge — do not edit manually.\n")
fmt.Fprintf(b, "# Re-generate: kforge gitea-actions > .gitea/workflows/deploy.yml\n") fmt.Fprintf(b, "# Re-generate: kforge gitea-actions > .gitea/workflows/deploy.yml\n")
fmt.Fprintf(b, "#\n") fmt.Fprintf(b, "#\n")
fmt.Fprintf(b, "# Required Gitea org secrets:\n") fmt.Fprintf(b, "# Required Gitea org secrets:\n")
fmt.Fprintf(b, "# DOCKER_USERNAME, DOCKER_PASSWORD, KFORGE_NODE_IP\n") fmt.Fprintf(b, "# DOCKER_USERNAME, DOCKER_PASSWORD\n")
fmt.Fprintf(b, "# CLOUDFLARE_API_TOKEN, CF_ZONE_ID_* (per zone)\n") fmt.Fprintf(b, "# KFORGE_NODE_IP (external-dns annotation target)\n")
fmt.Fprintf(b, "# SOPS_AGE_KEY\n")
fmt.Fprintf(b, "# Required Gitea repo secrets:\n") fmt.Fprintf(b, "# Required Gitea repo secrets:\n")
fmt.Fprintf(b, "# KUBE_HOST, KUBE_TOKEN, KUBE_CERTIFICATE\n") fmt.Fprintf(b, "# KUBE_HOST, KUBE_TOKEN, KUBE_CERTIFICATE\n")
fmt.Fprintf(b, "\n") fmt.Fprintf(b, "\n")
@@ -65,30 +63,22 @@ func writeGiteaHeader(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaAc
fmt.Fprintf(b, "\n") fmt.Fprintf(b, "\n")
} }
func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) { func writeDeployJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) {
fmt.Fprintf(b, "jobs:\n") fmt.Fprintf(b, "jobs:\n")
fmt.Fprintf(b, " build-and-deploy:\n") fmt.Fprintf(b, " build-and-deploy:\n")
fmt.Fprintf(b, " runs-on: ubuntu-latest\n") fmt.Fprintf(b, " runs-on: ubuntu-latest\n")
fmt.Fprintf(b, " steps:\n") fmt.Fprintf(b, " steps:\n")
// Checkout
writeStep(b, "Checkout", map[string]any{ writeStep(b, "Checkout", map[string]any{
"uses": "actions/checkout@v4", "uses": "actions/checkout@v4",
"with": map[string]any{"fetch-depth": 0}, "with": map[string]any{"fetch-depth": 0},
}) })
// Node (optional — only if package.json exists) writeStep(b, "Create short SHA", map[string]any{
writeStep(b, "Setup Node", map[string]any{
"uses": "actions/setup-node@v4",
"with": map[string]any{"node-version": opts.NodeVersion},
})
// Short SHA
writeStep(b, "Create short commit hash", map[string]any{
"run": `echo "SHORT_SHA=$(git rev-parse --short HEAD)" >> $GITHUB_ENV`, "run": `echo "SHORT_SHA=$(git rev-parse --short HEAD)" >> $GITHUB_ENV`,
}) })
// Docker login if !cfg.Registry.Insecure {
writeStep(b, "Login to registry", map[string]any{ writeStep(b, "Login to registry", map[string]any{
"uses": "docker/login-action@v2", "uses": "docker/login-action@v2",
"with": map[string]any{ "with": map[string]any{
@@ -97,12 +87,14 @@ func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActi
"password": "${{ secrets.DOCKER_PASSWORD }}", "password": "${{ secrets.DOCKER_PASSWORD }}",
}, },
}) })
}
writeStep(b, "Set up Docker Buildx", map[string]any{
"uses": "docker/setup-buildx-action@v3",
})
// Docker build + push
fullRepo := cfg.Registry.URL + "/" + cfg.Meta.Tenant + "/" + cfg.Meta.Name fullRepo := cfg.Registry.URL + "/" + cfg.Meta.Tenant + "/" + cfg.Meta.Name
writeStep(b, "Build and push image", map[string]any{ buildWith := map[string]any{
"uses": "docker/build-push-action@v5",
"with": map[string]any{
"context": ".", "context": ".",
"platforms": "linux/amd64", "platforms": "linux/amd64",
"file": cfg.Defaults.Dockerfile, "file": cfg.Defaults.Dockerfile,
@@ -110,15 +102,54 @@ func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActi
"tags": fmt.Sprintf("%s:latest\n%s:${{ env.SHORT_SHA }}", fullRepo, fullRepo), "tags": fmt.Sprintf("%s:latest\n%s:${{ env.SHORT_SHA }}", fullRepo, fullRepo),
"provenance": false, "provenance": false,
"sbom": false, "sbom": false,
}, }
if cfg.Registry.Insecure {
buildWith["buildkitd-config-inline"] = fmt.Sprintf(
"[registry.%q]\n http = true\n insecure = true",
cfg.Registry.URL,
)
}
writeStep(b, "Build and push image", map[string]any{
"uses": "docker/build-push-action@v5",
"with": buildWith,
}) })
// Install kforge on runner if cfg.ActionRef != "" {
writeActionDeploySteps(b, cfg, opts)
} else {
writeInlineDeploySteps(b, cfg, opts)
}
}
// writeActionDeploySteps emits one `uses: docker://image` step per environment.
// The docker:// prefix tells act/Gitea Actions to pull the image from the OCI
// registry directly, bypassing GitHub/Gitea source resolution.
func writeActionDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) {
image := "docker://" + cfg.ActionRef
for _, envKey := range opts.Environments {
env, err := config.ResolveEnvironment(cfg, envKey)
if err != nil {
continue
}
label := strings.Title(envKey) //nolint:staticcheck
writeStep(b, fmt.Sprintf("Deploy (%s)", label), map[string]any{
"uses": image,
"with": map[string]any{
"command": "deploy",
"env": envKey,
"namespace": env.Namespace,
},
"env": actionEnv(),
})
}
}
// writeInlineDeploySteps emits the classic multi-step inline approach.
func writeInlineDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) {
writeStep(b, "Install kforge", map[string]any{ writeStep(b, "Install kforge", map[string]any{
"run": "KFORGE_VERSION=\"latest\"\ncurl -fsSL \"https://kforge/releases/download/${KFORGE_VERSION}/kforge-linux-amd64\" -o /usr/local/bin/kforge\nchmod +x /usr/local/bin/kforge", "run": "KFORGE_VERSION=\"latest\"\ncurl -fsSL \"https://kforge/releases/download/${KFORGE_VERSION}/kforge-linux-amd64\" -o /usr/local/bin/kforge\nchmod +x /usr/local/bin/kforge",
}) })
// Per-environment deploy steps
for _, envKey := range opts.Environments { for _, envKey := range opts.Environments {
env, err := config.ResolveEnvironment(cfg, envKey) env, err := config.ResolveEnvironment(cfg, envKey)
if err != nil { if err != nil {
@@ -129,45 +160,26 @@ func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActi
} }
func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *config.ResolvedEnvironment, envKey string) { func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *config.ResolvedEnvironment, envKey string) {
label := strings.Title(envKey) //nolint:staticcheck // simple capitalisation label := strings.Title(envKey) //nolint:staticcheck
// Validate kforge config before doing anything destructive.
writeStep(b, fmt.Sprintf("Validate kforge config (%s)", label), map[string]any{ writeStep(b, fmt.Sprintf("Validate kforge config (%s)", label), map[string]any{
"run": "kforge validate", "run": "kforge validate",
"env": giteaSecretEnv(), "env": giteaNodeIPEnv(),
}) })
// Apply cluster secrets (only creates if missing — idempotent).
writeStep(b, fmt.Sprintf("Apply cluster secrets (%s)", label), map[string]any{ writeStep(b, fmt.Sprintf("Apply cluster secrets (%s)", label), map[string]any{
"run": fmt.Sprintf("kforge secrets apply --env %s", envKey), "run": fmt.Sprintf("kforge secrets apply --env %s", envKey),
"env": giteaKubeEnv(), "env": giteaKubeEnv(),
}) })
// DNS records
hasDNSHosts := false
for _, h := range env.Ingress.Hosts {
if h.DNSRecord {
hasDNSHosts = true
break
}
}
if hasDNSHosts && !cfg.DNS.SkipDNS {
writeStep(b, fmt.Sprintf("Ensure DNS records (%s)", label), map[string]any{
"run": fmt.Sprintf("kforge dns ensure --env %s", envKey),
"env": mergeMaps(giteaSecretEnv(), giteaKubeEnv()),
})
}
// Generate manifests
writeStep(b, fmt.Sprintf("Generate manifests (%s)", label), map[string]any{ writeStep(b, fmt.Sprintf("Generate manifests (%s)", label), map[string]any{
"run": fmt.Sprintf( "run": fmt.Sprintf(
"kforge generate --env %s --output .kforge-out --set image_tag=${{ env.SHORT_SHA }}", "kforge generate --env %s --output .kforge-out --set image_tag=${{ env.SHORT_SHA }}",
envKey, envKey,
), ),
"env": giteaSecretEnv(), "env": giteaNodeIPEnv(),
}) })
// kubectl apply
writeStep(b, fmt.Sprintf("Apply manifests (%s)", label), map[string]any{ writeStep(b, fmt.Sprintf("Apply manifests (%s)", label), map[string]any{
"uses": "actions-hub/kubectl@master", "uses": "actions-hub/kubectl@master",
"env": giteaKubeEnv(), "env": giteaKubeEnv(),
@@ -179,7 +191,6 @@ func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *conf
}, },
}) })
// Apply infra manifests if any infrastructure is enabled
infra := env.Infrastructure infra := env.Infrastructure
if infra.Database != nil || infra.Cache != nil || infra.Storage != nil || if infra.Database != nil || infra.Cache != nil || infra.Storage != nil ||
infra.Queue != nil || infra.Search != nil || infra.Monitoring != nil { infra.Queue != nil || infra.Search != nil || infra.Monitoring != nil {
@@ -195,7 +206,6 @@ func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *conf
}) })
} }
// Rollout restart
writeStep(b, fmt.Sprintf("Rollout restart (%s)", label), map[string]any{ writeStep(b, fmt.Sprintf("Rollout restart (%s)", label), map[string]any{
"uses": "actions-hub/kubectl@master", "uses": "actions-hub/kubectl@master",
"env": giteaKubeEnv(), "env": giteaKubeEnv(),
@@ -208,11 +218,182 @@ func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *conf
}) })
} }
// ------------------------------------------------------------
// Preview workflow
// ------------------------------------------------------------
// GeneratePreviewActions produces a Gitea Actions workflow YAML
// that deploys an ephemeral environment per pull request.
//
// On PR open/sync: builds a PR-tagged image, applies secrets,
// generates manifests (including Namespace), and deploys.
// On PR close: deletes the preview namespace, removing all resources.
func GeneratePreviewActions(cfg *config.KforgeConfig) (string, error) {
if !cfg.Preview.Enabled {
return "", fmt.Errorf("preview is not enabled in kforge.yml (set preview.enabled: true)")
}
nsPrefix := cfg.Preview.NamespacePrefix
if nsPrefix == "" {
nsPrefix = config.DefaultPreviewNamespacePrefix
}
fullRepo := cfg.Registry.URL + "/" + cfg.Meta.Tenant + "/" + cfg.Meta.Name
var b strings.Builder
fmt.Fprintf(&b, "# Generated by kforge — do not edit manually.\n")
fmt.Fprintf(&b, "# Re-generate: kforge gitea-preview > .gitea/workflows/preview.yml\n")
fmt.Fprintf(&b, "#\n")
fmt.Fprintf(&b, "# Required secrets:\n")
if !cfg.Registry.Insecure {
fmt.Fprintf(&b, "# DOCKER_USERNAME, DOCKER_PASSWORD (registry auth)\n")
}
fmt.Fprintf(&b, "# KFORGE_NODE_IP, KUBE_HOST, KUBE_TOKEN, KUBE_CERTIFICATE\n")
fmt.Fprintf(&b, "\n")
fmt.Fprintf(&b, "name: Preview Environment\n")
fmt.Fprintf(&b, "\n")
fmt.Fprintf(&b, "on:\n")
fmt.Fprintf(&b, " pull_request:\n")
fmt.Fprintf(&b, " types: [opened, synchronize, reopened, closed]\n")
fmt.Fprintf(&b, "\n")
fmt.Fprintf(&b, "jobs:\n")
fmt.Fprintf(&b, " preview:\n")
fmt.Fprintf(&b, " runs-on: ubuntu-latest\n")
fmt.Fprintf(&b, " steps:\n")
writeStep(&b, "Checkout", map[string]any{
"uses": "actions/checkout@v4",
"with": map[string]any{"fetch-depth": 0},
})
if !cfg.Registry.Insecure {
writeStep(&b, "Login to registry", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"uses": "docker/login-action@v2",
"with": map[string]any{
"registry": cfg.Registry.URL,
"username": "${{ secrets.DOCKER_USERNAME }}",
"password": "${{ secrets.DOCKER_PASSWORD }}",
},
})
}
writeStep(&b, "Set up Docker Buildx", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"uses": "docker/setup-buildx-action@v3",
})
previewBuildWith := map[string]any{
"context": ".",
"platforms": "linux/amd64",
"file": cfg.Defaults.Dockerfile,
"push": true,
"tags": fmt.Sprintf("%s:pr-${{ github.event.number }}", fullRepo),
"provenance": false,
"sbom": false,
}
if cfg.Registry.Insecure {
previewBuildWith["buildkitd-config-inline"] = fmt.Sprintf(
"[registry.%q]\n http = true\n insecure = true",
cfg.Registry.URL,
)
}
writeStep(&b, "Build and push preview image", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"uses": "docker/build-push-action@v5",
"with": previewBuildWith,
})
if cfg.ActionRef != "" {
image := "docker://" + cfg.ActionRef
writeStep(&b, "Deploy preview", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"uses": image,
"with": map[string]any{
"command": "preview-up",
"pr_number": "${{ github.event.number }}",
"namespace_prefix": nsPrefix,
},
"env": actionEnv(),
})
writeStep(&b, "Destroy preview", map[string]any{
"if": "${{ github.event.action == 'closed' }}",
"uses": image,
"with": map[string]any{
"command": "preview-down",
"pr_number": "${{ github.event.number }}",
"namespace_prefix": nsPrefix,
},
"env": actionEnv(),
})
} else {
writeStep(&b, "Install kforge", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"run": "KFORGE_VERSION=\"latest\"\ncurl -fsSL \"https://kforge/releases/download/${KFORGE_VERSION}/kforge-linux-amd64\" -o /usr/local/bin/kforge\nchmod +x /usr/local/bin/kforge",
})
writeStep(&b, "Apply preview secrets", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"run": "kforge secrets apply --pr-number ${{ github.event.number }}",
"env": giteaKubeEnv(),
})
writeStep(&b, "Generate preview manifests", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"run": "kforge generate --pr-number ${{ github.event.number }} --output .kforge-out",
"env": map[string]any{
"KFORGE_NODE_IP": "${{ secrets.KFORGE_NODE_IP }}",
"PR_NUMBER": "${{ github.event.number }}",
},
})
writeStep(&b, "Apply preview manifests", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"uses": "actions-hub/kubectl@master",
"env": giteaKubeEnv(),
"with": map[string]any{
"args": "apply -f .kforge-out/ --insecure-skip-tls-verify",
},
})
writeStep(&b, "Wait for preview rollout", map[string]any{
"if": "${{ github.event.action != 'closed' }}",
"uses": "actions-hub/kubectl@master",
"env": giteaKubeEnv(),
"with": map[string]any{
"args": fmt.Sprintf(
"rollout status deployment -n %s-${{ github.event.number }} --timeout=120s --insecure-skip-tls-verify",
nsPrefix,
),
},
})
writeStep(&b, "Destroy preview namespace", map[string]any{
"if": "${{ github.event.action == 'closed' }}",
"uses": "actions-hub/kubectl@master",
"env": giteaKubeEnv(),
"with": map[string]any{
"args": fmt.Sprintf(
"delete namespace %s-${{ github.event.number }} --ignore-not-found --insecure-skip-tls-verify",
nsPrefix,
),
},
})
}
return b.String(), nil
}
// ------------------------------------------------------------
// Shared step helpers
// ------------------------------------------------------------
// writeStep writes a single step in the jobs.steps list. // writeStep writes a single step in the jobs.steps list.
func writeStep(b *strings.Builder, name string, fields map[string]any) { func writeStep(b *strings.Builder, name string, fields map[string]any) {
fmt.Fprintf(b, "\n - name: %s\n", name) fmt.Fprintf(b, "\n - name: %s\n", name)
// Emit fields in a stable order. order := []string{"if", "uses", "run", "with", "env"}
order := []string{"uses", "run", "with", "env"}
for _, k := range order { for _, k := range order {
v, ok := fields[k] v, ok := fields[k]
if !ok { if !ok {
@@ -240,7 +421,6 @@ func writeStep(b *strings.Builder, name string, fields map[string]any) {
} }
func writeMapFields(b *strings.Builder, m map[string]any, indent string) { func writeMapFields(b *strings.Builder, m map[string]any, indent string) {
// Sort keys for stable output.
keys := make([]string, 0, len(m)) keys := make([]string, 0, len(m))
for k := range m { for k := range m {
keys = append(keys, k) keys = append(keys, k)
@@ -270,13 +450,11 @@ func writeMapFields(b *strings.Builder, m map[string]any, indent string) {
} }
} }
// giteaSecretEnv returns the env block referencing Gitea secrets // giteaNodeIPEnv returns env vars needed for kforge commands that
// needed for kforge itself (DNS, registry tokens, etc.). // resolve ${KFORGE_NODE_IP} tokens (validate, generate).
func giteaSecretEnv() map[string]any { func giteaNodeIPEnv() map[string]any {
return map[string]any{ return map[string]any{
"CLOUDFLARE_API_TOKEN": "${{ secrets.CLOUDFLARE_API_TOKEN }}",
"KFORGE_NODE_IP": "${{ secrets.KFORGE_NODE_IP }}", "KFORGE_NODE_IP": "${{ secrets.KFORGE_NODE_IP }}",
"SOPS_AGE_KEY": "${{ secrets.SOPS_AGE_KEY }}",
} }
} }
@@ -289,6 +467,17 @@ func giteaKubeEnv() map[string]any {
} }
} }
// actionEnv returns the combined env block for a kforge action step —
// kubectl auth plus the node IP for external-dns annotation resolution.
func actionEnv() map[string]any {
return map[string]any{
"KUBE_CERTIFICATE": "${{ secrets.KUBE_CERTIFICATE }}",
"KUBE_HOST": "${{ secrets.KUBE_HOST }}",
"KUBE_TOKEN": "${{ secrets.KUBE_TOKEN }}",
"KFORGE_NODE_IP": "${{ secrets.KFORGE_NODE_IP }}",
}
}
func mergeMaps(maps ...map[string]any) map[string]any { func mergeMaps(maps ...map[string]any) map[string]any {
result := map[string]any{} result := map[string]any{}
for _, m := range maps { for _, m := range maps {
@@ -301,8 +490,6 @@ func mergeMaps(maps ...map[string]any) map[string]any {
func sortedEnvKeys(cfg *config.KforgeConfig) []string { func sortedEnvKeys(cfg *config.KforgeConfig) []string {
keys := config.EnvironmentKeys(cfg) keys := config.EnvironmentKeys(cfg)
// Put staging/dev before production — a simple heuristic that
// matches the most common deploy order.
priority := map[string]int{"dev": 0, "development": 0, "staging": 1, "production": 2, "prod": 2} priority := map[string]int{"dev": 0, "development": 0, "staging": 1, "production": 2, "prod": 2}
sort.Slice(keys, func(i, j int) bool { sort.Slice(keys, func(i, j int) bool {
pi, pj := priority[keys[i]], priority[keys[j]] pi, pj := priority[keys[i]], priority[keys[j]]
+79 -27
View File
@@ -2,7 +2,6 @@ package generator
import ( import (
"fmt" "fmt"
"strings"
"kforge/internal/config" "kforge/internal/config"
"kforge/pkg/interpolate" "kforge/pkg/interpolate"
@@ -79,18 +78,22 @@ type InfraManifest struct {
} }
// ------------------------------------------------------------ // ------------------------------------------------------------
// Database — CNPG // Database — CNPG (centralized cluster)
// ------------------------------------------------------------ // ------------------------------------------------------------
func generateDatabase(env *config.ResolvedEnvironment, db *config.DatabaseInfraConfig) ([]InfraManifest, error) { func generateDatabase(env *config.ResolvedEnvironment, db *config.DatabaseInfraConfig) ([]InfraManifest, error) {
dbName := db.DatabaseName dbName := db.DatabaseName
if dbName == "" { if dbName == "" {
dbName = interpolate.Slug(env.FullName) dbName = interpolate.PGIdentifier(env.FullName)
} }
roleName := dbName + "_role" // pgUser must be a valid unquoted PostgreSQL identifier; same as
// the username stored in the db-credentials Secret by `kforge secrets apply`.
pgUser := interpolate.PGIdentifier(env.FullName)
secretName := env.FullName + "-db-credentials" secretName := env.FullName + "-db-credentials"
cnpgHost := env.CNPGHost
// CNPG Database CR // CNPG Database CR — declaratively manages the database lifecycle.
// The owner role is created by the db-init Job below.
dbManifest := fmt.Sprintf(`apiVersion: postgresql.cnpg.io/v1 dbManifest := fmt.Sprintf(`apiVersion: postgresql.cnpg.io/v1
kind: Database kind: Database
metadata: metadata:
@@ -103,47 +106,96 @@ spec:
name: %s name: %s
owner: %s owner: %s
cluster: cluster:
name: cnpg-main
`, dbName, env.Namespace, env.FullName, dbName, roleName)
// CNPG Role CR — CNPG creates and rotates the password,
// storing it in the secret named below.
roleManifest := fmt.Sprintf(`apiVersion: postgresql.cnpg.io/v1
kind: DatabaseRole
metadata:
name: %s name: %s
`, dbName, env.Namespace, env.FullName, dbName, pgUser, env.CNPGClusterName)
// db-init Job — runs on every deploy to ensure the PostgreSQL role
// exists and its password matches the Secret. The password stored by
// `kforge secrets apply` is alphanumeric-only so it is safe to
// embed in a shell command without additional escaping.
//
// Prerequisites:
// - The CNPG superuser Secret must exist in the same namespace
// (or copy it there as part of cluster bootstrap).
// - `kforge secrets apply` must have run before this Job.
jobManifest := fmt.Sprintf(`apiVersion: batch/v1
kind: Job
metadata:
name: %s-db-init
namespace: %s namespace: %s
labels: labels:
app: %s app: %s
managed-by: kforge managed-by: kforge
spec: spec:
ttlSecondsAfterFinished: 600
template:
metadata:
labels:
app: %s
spec:
restartPolicy: OnFailure
containers:
- name: db-init
image: postgres:16-alpine
command:
- /bin/sh
- -c
- |
set -e
echo "Ensuring role $DB_USER exists..."
PGPASSWORD="$ADMIN_PASSWORD" psql -h "$DB_HOST" -U "$ADMIN_USER" postgres \
-c "SELECT 1 FROM pg_roles WHERE rolname = '$DB_USER'" | grep -q 1 || \
PGPASSWORD="$ADMIN_PASSWORD" psql -h "$DB_HOST" -U "$ADMIN_USER" postgres \
-c "CREATE ROLE $DB_USER WITH LOGIN PASSWORD '$DB_PASSWORD';"
echo "Syncing password for $DB_USER..."
PGPASSWORD="$ADMIN_PASSWORD" psql -h "$DB_HOST" -U "$ADMIN_USER" postgres \
-c "ALTER ROLE $DB_USER WITH PASSWORD '$DB_PASSWORD';"
echo "Done."
env:
- name: DB_HOST
value: %s
- name: DB_USER
valueFrom:
secretKeyRef:
name: %s name: %s
passwordSecret: key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: %s name: %s
login: true key: password
superuser: false - name: ADMIN_USER
createdb: false valueFrom:
`, roleName, env.Namespace, env.FullName, roleName, secretName) secretKeyRef:
name: %s
// The env vars reference the CNPG-managed secret. key: username
// CNPG populates: username, password keys in the secret. - name: ADMIN_PASSWORD
// We assemble DATABASE_URL from the known CNPG host + db name. valueFrom:
cnpgHost := env.CNPGHost secretKeyRef:
dbURL := fmt.Sprintf("postgresql://$(%s_USER):$(%s_PASSWORD)@%s/%s", name: %s
strings.ToUpper(env.FullName), strings.ToUpper(env.FullName), cnpgHost, dbName) key: password
`,
env.FullName, env.Namespace, env.FullName, env.FullName,
cnpgHost,
secretName, secretName,
env.CNPGSuperuserSecret, env.CNPGSuperuserSecret,
)
// DATABASE_URL uses Kubernetes $(VAR_NAME) substitution — DB_USER and
// DB_PASSWORD must be defined earlier in the env list.
envVars := []config.EnvVarConfig{ envVars := []config.EnvVarConfig{
{Name: "DB_HOST", Type: config.EnvVarTypePlain, Value: cnpgHost}, {Name: "DB_HOST", Type: config.EnvVarTypePlain, Value: cnpgHost},
{Name: "DB_PORT", Type: config.EnvVarTypePlain, Value: "5432"}, {Name: "DB_PORT", Type: config.EnvVarTypePlain, Value: "5432"},
{Name: "DB_NAME", Type: config.EnvVarTypePlain, Value: dbName}, {Name: "DB_NAME", Type: config.EnvVarTypePlain, Value: dbName},
{Name: "DB_USER", Type: config.EnvVarTypeSecretRef, SecretName: secretName, SecretKey: "username"}, {Name: "DB_USER", Type: config.EnvVarTypeSecretRef, SecretName: secretName, SecretKey: "username"},
{Name: "DB_PASSWORD", Type: config.EnvVarTypeSecretRef, SecretName: secretName, SecretKey: "password"}, {Name: "DB_PASSWORD", Type: config.EnvVarTypeSecretRef, SecretName: secretName, SecretKey: "password"},
{Name: "DATABASE_URL", Type: config.EnvVarTypePlain, Value: dbURL}, {Name: "DATABASE_URL", Type: config.EnvVarTypePlain,
Value: fmt.Sprintf("postgresql://$(DB_USER):$(DB_PASSWORD)@%s/%s", cnpgHost, dbName)},
} }
return []InfraManifest{ return []InfraManifest{
{Name: "cnpg-database", Content: dbManifest, EnvVars: envVars}, {Name: "cnpg-database", Content: dbManifest, EnvVars: envVars},
{Name: "cnpg-role", Content: roleManifest}, {Name: "cnpg-db-init", Content: jobManifest},
}, nil }, nil
} }
+40
View File
@@ -151,8 +151,10 @@ func Deployment(env *config.ResolvedEnvironment, tokens interpolate.Tokens) (str
b.WriteString(renderResourceLines(env.Resources, " ")) b.WriteString(renderResourceLines(env.Resources, " "))
if env.ImagePullSecret != "" {
b.WriteString(" imagePullSecrets:\n") b.WriteString(" imagePullSecrets:\n")
fmt.Fprintf(&b, " - name: %s\n", env.ImagePullSecret) fmt.Fprintf(&b, " - name: %s\n", env.ImagePullSecret)
}
return b.String(), nil return b.String(), nil
} }
@@ -181,6 +183,22 @@ func Ingress(env *config.ResolvedEnvironment, _ interpolate.Tokens) (string, err
b.WriteString(" nginx.ingress.kubernetes.io/auth-realm: \"Authentication Required\"\n") b.WriteString(" nginx.ingress.kubernetes.io/auth-realm: \"Authentication Required\"\n")
} }
// external-dns annotations: collect all hostnames that need DNS records.
if !env.SkipDNS {
var dnsHosts []string
for _, h := range env.Ingress.Hosts {
if h.DNSRecord {
dnsHosts = append(dnsHosts, h.Hostname)
}
}
if len(dnsHosts) > 0 {
fmt.Fprintf(&b, " external-dns.alpha.kubernetes.io/hostname: \"%s\"\n", strings.Join(dnsHosts, ","))
if env.DNSTarget != "" {
fmt.Fprintf(&b, " external-dns.alpha.kubernetes.io/target: \"%s\"\n", env.DNSTarget)
}
}
}
b.WriteString("spec:\n") b.WriteString("spec:\n")
fmt.Fprintf(&b, " ingressClassName: %s\n", env.IngressClass) fmt.Fprintf(&b, " ingressClassName: %s\n", env.IngressClass)
@@ -315,8 +333,10 @@ func CronJob(env *config.ResolvedEnvironment, job *config.ResolvedCronJob, token
b.WriteString(renderResourceLines(*job.Resources, " ")) b.WriteString(renderResourceLines(*job.Resources, " "))
} }
if env.ImagePullSecret != "" {
b.WriteString(" imagePullSecrets:\n") b.WriteString(" imagePullSecrets:\n")
fmt.Fprintf(&b, " - name: %s\n", env.ImagePullSecret) fmt.Fprintf(&b, " - name: %s\n", env.ImagePullSecret)
}
return b.String(), nil return b.String(), nil
} }
@@ -405,6 +425,26 @@ func renderStrSlice(ss []string) string {
return "[" + strings.Join(quoted, ", ") + "]" return "[" + strings.Join(quoted, ", ") + "]"
} }
// Namespace generates a Namespace manifest. Used for preview
// environments where the namespace must be created by kubectl apply.
func Namespace(namespace string, labels map[string]string) string {
var b strings.Builder
b.WriteString("apiVersion: v1\nkind: Namespace\nmetadata:\n")
fmt.Fprintf(&b, " name: %s\n", namespace)
if len(labels) > 0 {
b.WriteString(" labels:\n")
// Stable key order.
keys := make([]string, 0, len(labels))
for k := range labels {
keys = append(keys, k)
}
for _, k := range keys {
fmt.Fprintf(&b, " %s: %q\n", k, labels[k])
}
}
return b.String()
}
// render executes a template with data and returns the result. // render executes a template with data and returns the result.
func render(t *template.Template, data any) (string, error) { func render(t *template.Template, data any) (string, error) {
var buf bytes.Buffer var buf bytes.Buffer
+3 -10
View File
@@ -3,18 +3,11 @@ meta:
tenant: nate-lubitz tenant: nate-lubitz
registry: registry:
url: registry.natelubitz.com url: registry.container-registry.svc.cluster.local:5000
pull_secret: regcred insecure: true
dns: dns:
provider: cloudflare target: ${KFORGE_NODE_IP}
cloudflare:
api_token: ${CLOUDFLARE_API_TOKEN}
zones:
- name: natelubitz.com
zone_id: ${CF_ZONE_ID_NATELUBITZ}
proxied: false
node_ip: ${KFORGE_NODE_IP}
cluster: cluster:
tls_issuer: letsencrypt-prod tls_issuer: letsencrypt-prod
+17
View File
@@ -119,3 +119,20 @@ func Slug(s string) string {
func HostSlug(hostname string) string { func HostSlug(hostname string) string {
return Slug(strings.ReplaceAll(hostname, ".", "-")) return Slug(strings.ReplaceAll(hostname, ".", "-"))
} }
// PGIdentifier converts a kforge name into a valid unquoted
// PostgreSQL identifier: lowercase, hyphens and spaces become
// underscores, all other non-alphanumeric characters are dropped.
// "prod-my-tenant-myapp" → "prod_my_tenant_myapp"
func PGIdentifier(s string) string {
var b strings.Builder
for _, r := range strings.ToLower(s) {
switch {
case r >= 'a' && r <= 'z', r >= '0' && r <= '9', r == '_':
b.WriteRune(r)
case r == '-', r == ' ':
b.WriteRune('_')
}
}
return strings.Trim(b.String(), "_")
}
+199 -71
View File
@@ -1,8 +1,8 @@
# kforge # kforge
**kforge** eliminates Kubernetes boilerplate. You define your app once in a `kforge.yml` file — environments, infrastructure, ingress, TLS, DNS — and kforge generates production-ready flat manifests on every CI run. Nothing is committed to your repo except the config. **kforge** eliminates Kubernetes boilerplate. You define your app once in a `kforge.yml` file — environments, infrastructure, ingress, TLS, DNS — and kforge generates production-ready flat manifests on every CI run. Nothing is committed to your repo except the config and generated workflows.
Built for self-hosted MicroK8s, Gitea Actions, Cloudflare DNS, cert-manager, and CNPG — but designed to be extended. Built for self-hosted MicroK8s, Gitea Actions, external-dns, cert-manager, and CNPG — but designed to be extended.
--- ---
@@ -10,12 +10,14 @@ Built for self-hosted MicroK8s, Gitea Actions, Cloudflare DNS, cert-manager, and
1. Add a `kforge.yml` to your repo root describing your app, environments, and infrastructure. 1. Add a `kforge.yml` to your repo root describing your app, environments, and infrastructure.
2. kforge generates Kubernetes YAML at CI time — Service, Deployment, Ingress, cert-manager Certificates, CronJobs, and infrastructure (database, cache, storage, queue, search). 2. kforge generates Kubernetes YAML at CI time — Service, Deployment, Ingress, cert-manager Certificates, CronJobs, and infrastructure (database, cache, storage, queue, search).
3. Generated manifests are applied to the cluster and discarded. Only `kforge.yml` is committed. 3. Generated manifests are applied to the cluster and discarded. Only `kforge.yml` and generated workflow files are committed.
``` ```
kforge.yml → kforge generate → kubectl apply → cluster kforge.yml → kforge generate → kubectl apply → cluster
``` ```
DNS records are managed by **external-dns** running in the cluster — kforge writes the appropriate annotations on the Ingress and external-dns creates the records automatically. TLS is handled by **cert-manager** reading those same Certificate CRs.
--- ---
## Installation ## Installation
@@ -57,13 +59,7 @@ registry:
url: registry.yourdomain.com url: registry.yourdomain.com
dns: dns:
provider: cloudflare target: ${KFORGE_NODE_IP} # your cluster node's public IP
cloudflare:
api_token: ${CLOUDFLARE_API_TOKEN}
zones:
- name: yourdomain.com
zone_id: ${CF_ZONE_ID_YOURDOMAIN}
node_ip: ${KFORGE_NODE_IP}
cluster: cluster:
tls_issuer: letsencrypt-prod tls_issuer: letsencrypt-prod
@@ -94,7 +90,7 @@ environments:
hosts: hosts:
- hostname: app-staging.yourdomain.com - hostname: app-staging.yourdomain.com
tls: true tls: true
dns_record: true dns_record: true # external-dns creates this record
auth: auth:
enabled: true enabled: true
users: users:
@@ -131,19 +127,101 @@ kforge secrets list
kforge generate --env production --dry-run kforge generate --env production --dry-run
``` ```
**4. Generate your Gitea Actions workflow:** **4. Generate your Gitea Actions workflows:**
```bash ```bash
kforge gitea-actions > .gitea/workflows/deploy.yml kforge gitea-actions > .gitea/workflows/deploy.yml
kforge gitea-preview > .gitea/workflows/preview.yml # optional — PR preview environments
``` ```
--- ---
## Adding kforge to an existing project
### Step 1 — Add `kforge.yml` and generate workflows
Add `kforge.yml` to your repo root (see Quick start above), then:
```bash
kforge gitea-actions # → .gitea/workflows/deploy.yml
kforge gitea-preview # → .gitea/workflows/preview.yml (if preview.enabled: true)
```
Commit `kforge.yml` and both generated workflow files.
### Step 2 — Gitea secrets
**Org-level** (set once, shared across all repos):
| Secret | Value |
|---|---|
| `DOCKER_USERNAME` | Registry login username |
| `DOCKER_PASSWORD` | Registry login password/token |
| `KFORGE_NODE_IP` | Your cluster's public node IP (used as the external-dns record target) |
**Repo-level** (per project):
| Secret | How to get it |
|---|---|
| `KUBE_HOST` | `kubectl config view --raw -o jsonpath='{.clusters[0].cluster.server}'` |
| `KUBE_CERTIFICATE` | `kubectl config view --raw -o jsonpath='{.clusters[0].cluster.certificate-authority-data}'` |
| `KUBE_TOKEN` | See below |
Create a long-lived deploy service account:
```bash
kubectl create serviceaccount kforge-deployer -n kube-system
kubectl create clusterrolebinding kforge-deployer \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:kforge-deployer
kubectl create token kforge-deployer -n kube-system --duration=8760h
```
### Step 3 — Cluster prerequisites
These must already be running on your cluster:
| Component | Purpose |
|---|---|
| **NGINX ingress controller** | Handles `ingress_class: nginx` |
| **cert-manager** + `ClusterIssuer` named `letsencrypt-prod` | Issues TLS certificates |
| **external-dns** | Reads Ingress annotations, creates DNS records |
| **CNPG operator** | Required if `infrastructure.database` is enabled |
| **`regcred` imagePullSecret** | Must exist in each environment namespace |
For external-dns, configure it with your DNS provider and `--source=ingress`. kforge writes the target annotation automatically from `dns.target`.
Create `regcred` in each namespace:
```bash
kubectl create secret docker-registry regcred \
--docker-server=registry.yourdomain.com \
--docker-username=<user> \
--docker-password=<pass> \
-n production
```
### Step 4 — First deploy
The workflow is fully automated after setup, but the very first time:
1. Create the namespace: `kubectl create namespace production`
2. Create `regcred` in that namespace (above).
3. If using CNPG database, make the CNPG superuser Secret available in the namespace. The default secret name is `cnpg-main-superuser` — copy it from wherever your CNPG cluster lives:
```bash
kubectl get secret cnpg-main-superuser -n cnpg-system -o yaml \
| sed 's/namespace: cnpg-system/namespace: production/' \
| kubectl apply -f -
```
4. Push to `main`. The workflow runs `kforge secrets apply` (creates credentials), `kforge generate` (writes manifests), and `kubectl apply`.
---
## CLI reference ## CLI reference
### `kforge validate` ### `kforge validate`
Parses `kforge.yml`, checks structural correctness, and verifies all required secrets are present in the current environment. Exits non-zero if anything is wrong — use this as the first step in CI to fail fast before touching the cluster. Parses `kforge.yml`, checks structural correctness, and verifies all required secrets are present in the current environment. Use this as the first step in CI to fail fast before touching the cluster.
```bash ```bash
kforge validate kforge validate
@@ -161,15 +239,16 @@ kforge generate # all environments
kforge generate --env staging # one environment kforge generate --env staging # one environment
kforge generate --env production --dry-run kforge generate --env production --dry-run
kforge generate --env production --output .kube/ kforge generate --env production --output .kube/
kforge generate --pr-number 42 # preview environment for PR #42
``` ```
**Output files per environment:** **Output files per environment:**
| File | Contents | | File | Contents |
| --------------------------------- | ---------------------------------------------------- | |---|---|
| `{env}-core.yaml` | Service, Deployment, Ingress, Certificates, CronJobs | | `{env}-core.yaml` | Service, Deployment, Ingress, Certificates, CronJobs |
| `{env}-infra-cnpg-database.yaml` | CNPG Database CR | | `{env}-infra-cnpg-database.yaml` | CNPG Database CR |
| `{env}-infra-cnpg-role.yaml` | CNPG DatabaseRole CR | | `{env}-infra-cnpg-db-init.yaml` | Job that creates the PostgreSQL role and syncs password |
| `{env}-infra-cache.yaml` | Valkey/Redis Deployment or StatefulSet | | `{env}-infra-cache.yaml` | Valkey/Redis Deployment or StatefulSet |
| `{env}-infra-cache-svc.yaml` | Cache Service | | `{env}-infra-cache-svc.yaml` | Cache Service |
| `{env}-infra-storage.yaml` | Minio Deployment | | `{env}-infra-storage.yaml` | Minio Deployment |
@@ -177,6 +256,8 @@ kforge generate --env production --output .kube/
| `{env}-infra-search.yaml` | Meilisearch Deployment | | `{env}-infra-search.yaml` | Meilisearch Deployment |
| `{env}-infra-servicemonitor.yaml` | Prometheus ServiceMonitor CR | | `{env}-infra-servicemonitor.yaml` | Prometheus ServiceMonitor CR |
When `--pr-number` is given, the output also includes a `Namespace` manifest so `kubectl apply` is self-contained — no separate namespace creation step needed.
Infrastructure env vars (`DATABASE_URL`, `CACHE_URL`, `STORAGE_ENDPOINT`, etc.) are automatically injected into the Deployment — you don't wire these up manually. Infrastructure env vars (`DATABASE_URL`, `CACHE_URL`, `STORAGE_ENDPOINT`, etc.) are automatically injected into the Deployment — you don't wire these up manually.
--- ---
@@ -194,10 +275,7 @@ Example output:
``` ```
── Gitea org secret ── ── Gitea org secret ──
DOCKER_USERNAME ✗ missing DOCKER_USERNAME ✗ missing
CLOUDFLARE_API_TOKEN ✓ set
CF_ZONE_ID_YOURDOMAIN_COM ✓ set
KFORGE_NODE_IP ✓ set KFORGE_NODE_IP ✓ set
SOPS_AGE_KEY ✗ missing
── Gitea repo secret ── ── Gitea repo secret ──
KUBE_HOST ✓ set KUBE_HOST ✓ set
@@ -218,27 +296,26 @@ Generates secure random credentials and creates Kubernetes Secrets in the cluste
```bash ```bash
kforge secrets apply --env staging kforge secrets apply --env staging
kforge secrets apply --env production --force # rotates all credentials kforge secrets apply --env production --force # rotates all credentials
kforge secrets apply --pr-number 42 # apply secrets for PR preview #42
``` ```
For basic auth secrets, kforge prints the generated passwords once at apply time. Save them — they are not stored anywhere else. **What gets created:**
| Secret | Contents |
|---|---|
| `{full_name}-db-credentials` | PostgreSQL `username` (derived from app name) + random alphanumeric `password` |
| `{full_name}-basic-auth` | htpasswd entries for ingress basic auth |
| `{full_name}-cache-credentials` | Valkey/Redis password |
| `{full_name}-storage-credentials` | Minio access key + secret key |
| `{full_name}-queue-credentials` | RabbitMQ username + password (NATS needs no credentials) |
| `{full_name}-search-credentials` | Meilisearch master key |
For basic auth secrets, kforge prints the generated passwords once at apply time — save them.
Requires `KUBE_HOST`, `KUBE_TOKEN`, and `KUBE_CERTIFICATE` to be set (Gitea injects these automatically during CI). Requires `KUBE_HOST`, `KUBE_TOKEN`, and `KUBE_CERTIFICATE` to be set (Gitea injects these automatically during CI).
--- ---
### `kforge dns ensure`
Creates or updates Cloudflare DNS A records for all ingress hosts with `dns_record: true`. Idempotent — no-ops if the record already points to the correct IP.
```bash
kforge dns ensure --env staging
kforge dns ensure --env staging --env production
```
Requires `CLOUDFLARE_API_TOKEN` and `KFORGE_NODE_IP` to be set.
---
### `kforge gitea-actions` ### `kforge gitea-actions`
Generates a complete `.gitea/workflows/deploy.yml` for this app. Re-run whenever you add environments or change deploy configuration. Generates a complete `.gitea/workflows/deploy.yml` for this app. Re-run whenever you add environments or change deploy configuration.
@@ -251,14 +328,32 @@ kforge gitea-actions --branch main --env staging --env production
The generated workflow runs these steps for each environment, in order: The generated workflow runs these steps for each environment, in order:
1. Build and push Docker image 1. Build and push Docker image (tagged with git SHA)
2. `kforge validate` 2. `kforge validate`
3. `kforge secrets apply` — creates missing cluster secrets 3. `kforge secrets apply` — creates any missing cluster secrets
4. `kforge dns ensure` — creates missing DNS records 4. `kforge generate` — writes manifests to `.kforge-out/`
5. `kforge generate` — writes manifests to `.kforge-out/` 5. `kubectl apply` — applies core manifests (Service, Deployment, Ingress, Certs)
6. `kubectl apply` — applies core manifests 6. `kubectl apply` — applies infrastructure manifests (database, cache, etc.)
7. `kubectl apply` — applies infra manifests 7. `kubectl rollout restart` — triggers rolling update
8. `kubectl rollout restart` — triggers rolling update
DNS records are created automatically by external-dns when the Ingress is applied — no separate DNS step needed.
---
### `kforge gitea-preview`
Generates `.gitea/workflows/preview.yml` that deploys an ephemeral environment for each pull request.
```bash
kforge gitea-preview
kforge gitea-preview --output .gitea/workflows/preview.yml
```
Requires `preview.enabled: true` in `kforge.yml`. See the [preview environments](#preview-environments) section below.
The generated workflow:
- **On PR open / sync**: Builds a `:pr-{N}` tagged image, applies secrets to a new namespace (`preview-pr-{N}`), generates manifests (including a `Namespace` resource), and deploys.
- **On PR close**: Deletes the `preview-pr-{N}` namespace, removing all preview resources automatically.
--- ---
@@ -269,7 +364,7 @@ The generated workflow runs these steps for each environment, in order:
kforge generates resource names using the pattern `{env_prefix}-{tenant}-{name}` (e.g. `prod-my-org-my-app`). Use `${tokens}` anywhere in string values to reference resolved fields: kforge generates resource names using the pattern `{env_prefix}-{tenant}-{name}` (e.g. `prod-my-org-my-app`). Use `${tokens}` anywhere in string values to reference resolved fields:
| Token | Resolves to | | Token | Resolves to |
| --------------- | ------------------------------------------- | |---|---|
| `${name}` | `meta.name` | | `${name}` | `meta.name` |
| `${tenant}` | `meta.tenant` | | `${tenant}` | `meta.tenant` |
| `${env}` | current environment key | | `${env}` | current environment key |
@@ -277,7 +372,7 @@ kforge generates resource names using the pattern `{env_prefix}-{tenant}-{name}`
| `${full_name}` | `{env_prefix}-{tenant}-{name}` | | `${full_name}` | `{env_prefix}-{tenant}-{name}` |
| `${namespace}` | resolved namespace for the environment | | `${namespace}` | resolved namespace for the environment |
Environment variables (e.g. `${CLOUDFLARE_API_TOKEN}`) are resolved from the CI process environment at generation time — never hardcode secrets in `kforge.yml`. Environment variables (e.g. `${KFORGE_NODE_IP}`) are resolved from the CI process environment at generation time — never hardcode secrets in `kforge.yml`.
--- ---
@@ -308,20 +403,20 @@ Override per-environment by adding a `registry:` block under the environment.
### `dns` ### `dns`
```yaml DNS records are managed by **external-dns** inside the cluster. kforge writes annotations on the Ingress resource for each host with `dns_record: true`:
dns:
provider: cloudflare # currently supported: cloudflare ```
cloudflare: external-dns.alpha.kubernetes.io/hostname: "app.yourdomain.com"
api_token: ${CLOUDFLARE_API_TOKEN} external-dns.alpha.kubernetes.io/target: "<dns.target>"
zones:
- name: yourdomain.com
zone_id: ${CF_ZONE_ID_YOURDOMAIN}
proxied: false # false = DNS-only, required for cert-manager DNS-01
node_ip: ${KFORGE_NODE_IP} # IP for new A records
skip_dns: false # true to disable all DNS management
``` ```
kforge matches each ingress hostname to the correct zone by longest-suffix match — add one zone entry per domain you own. ```yaml
dns:
target: ${KFORGE_NODE_IP} # value for the external-dns target annotation
skip_dns: false # true = don't write external-dns annotations
```
`target` supports token interpolation — `${KFORGE_NODE_IP}` is the most common value, resolved from the `KFORGE_NODE_IP` Gitea org secret at generate time.
--- ---
@@ -332,7 +427,9 @@ cluster:
tls_issuer: letsencrypt-prod # default: letsencrypt-prod tls_issuer: letsencrypt-prod # default: letsencrypt-prod
ingress_class: nginx # default: nginx ingress_class: nginx # default: nginx
cnpg: cnpg:
host: cnpg-main-rw.default.svc.cluster.local # your CNPG cluster service host: cnpg-main-rw.default.svc.cluster.local # your CNPG cluster's read-write service
cluster_name: cnpg-main # default: cnpg-main
superuser_secret: cnpg-main-superuser # default: cnpg-main-superuser
namespace_pattern: "${env}" # default: environment key namespace_pattern: "${env}" # default: environment key
``` ```
@@ -421,7 +518,7 @@ environments:
**Injected env vars per service** (automatically added to your Deployment): **Injected env vars per service** (automatically added to your Deployment):
| Service | Env vars injected | | Service | Env vars injected |
| ---------------- | ------------------------------------------------------------------------- | |---|---|
| database (CNPG) | `DATABASE_URL`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` | | database (CNPG) | `DATABASE_URL`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` |
| cache | `CACHE_URL`, `CACHE_HOST`, `CACHE_PORT`, `CACHE_PASSWORD` | | cache | `CACHE_URL`, `CACHE_HOST`, `CACHE_PORT`, `CACHE_PASSWORD` |
| storage | `STORAGE_ENDPOINT`, `STORAGE_ACCESS_KEY`, `STORAGE_SECRET_KEY` | | storage | `STORAGE_ENDPOINT`, `STORAGE_ACCESS_KEY`, `STORAGE_SECRET_KEY` |
@@ -429,6 +526,40 @@ environments:
| queue (RabbitMQ) | `QUEUE_URL`, `QUEUE_USER`, `QUEUE_PASSWORD` | | queue (RabbitMQ) | `QUEUE_URL`, `QUEUE_USER`, `QUEUE_PASSWORD` |
| search | `SEARCH_URL`, `SEARCH_MASTER_KEY` | | search | `SEARCH_URL`, `SEARCH_MASTER_KEY` |
#### CNPG database details
For a centralized CNPG cluster, kforge generates two resources:
1. A **`Database` CR** — declaratively manages the database lifecycle (CNPG v1.22+).
2. A **`db-init` Job** — runs on every deploy to create the PostgreSQL role (if it doesn't exist) and sync its password from the `{full_name}-db-credentials` Secret.
`kforge secrets apply` must run before the Job so the Secret exists. The PostgreSQL username is derived from the app's full name (`prod_my_org_my_app`). The password is alphanumeric only, which keeps the init Job shell script simple and safe.
The `cnpg-main-superuser` Secret (created by CNPG for the cluster) must be present in the target namespace. Copy it once during cluster bootstrap or namespace creation.
---
### `preview`
Enables PR preview environments. Run `kforge gitea-preview` to generate the workflow.
```yaml
preview:
enabled: true
base_environment: staging # inherit infra and env vars from this env
namespace_prefix: preview-pr # namespace = preview-pr-{PR_NUMBER}
hostname_template: "pr-${PR_NUMBER}.${name}.yourdomain.com"
```
Supported tokens in `hostname_template`: `${PR_NUMBER}`, `${name}`, `${tenant}`.
On PR open/sync, kforge deploys the app to a `preview-pr-{N}` namespace with:
- Image tagged `:pr-{N}` (built from the PR branch)
- Hostname from the template
- Infrastructure and env vars inherited from `base_environment`
On PR close, the entire namespace is deleted.
--- ---
### `environments` ### `environments`
@@ -460,7 +591,7 @@ environments:
hosts: hosts:
- hostname: app.yourdomain.com - hostname: app.yourdomain.com
tls: true # kforge generates a cert-manager Certificate tls: true # kforge generates a cert-manager Certificate
dns_record: true # kforge creates a Cloudflare A record dns_record: true # external-dns creates this record
auth: auth:
enabled: false # enable for staging/dev to protect unreleased work enabled: false # enable for staging/dev to protect unreleased work
users: users:
@@ -493,7 +624,7 @@ environments:
lifecycle: lifecycle:
delete: false # if true + previous_name set, deletes old resources delete: false # if true + previous_name set, deletes old resources
delete_grace_seconds: 300 # 5-minute countdown before deletion runs in CI delete_grace_seconds: 300
``` ```
--- ---
@@ -506,21 +637,20 @@ kforge works with three categories of secrets, each living in the right place fo
Set once at the organisation level; available to every repo automatically. Set once at the organisation level; available to every repo automatically.
| Secret | Purpose | | Secret | Used by | Purpose |
| ---------------------- | ---------------------------------------------- | |---|---|---|
| `DOCKER_USERNAME` | Registry authentication | | `DOCKER_USERNAME` | `docker/login-action` | Registry authentication for image push |
| `DOCKER_PASSWORD` | Registry authentication | | `DOCKER_PASSWORD` | `docker/login-action` | Registry authentication for image push |
| `CLOUDFLARE_API_TOKEN` | DNS record management (Zone:Read + DNS:Edit) | | `KFORGE_NODE_IP` | kforge | Cluster node IP — written as the external-dns target annotation |
| `CF_ZONE_ID_{DOMAIN}` | One per zone, e.g. `CF_ZONE_ID_YOURDOMAIN_COM` |
| `KFORGE_NODE_IP` | MicroK8s node IP for DNS A records | `DOCKER_USERNAME` and `DOCKER_PASSWORD` are consumed by the Docker build steps in the generated workflow, not by kforge itself. `kforge validate` does not check for them.
| `SOPS_AGE_KEY` | Decrypts `.kforge/secrets.enc.yml` |
### Category B — Gitea repo secrets ### Category B — Gitea repo secrets
Per-repo, since different apps may deploy to different clusters. Per-repo, since different apps may deploy to different clusters.
| Secret | Purpose | | Secret | Purpose |
| ------------------ | ----------------------------- | |---|---|
| `KUBE_HOST` | Kubernetes API server URL | | `KUBE_HOST` | Kubernetes API server URL |
| `KUBE_TOKEN` | Service account token | | `KUBE_TOKEN` | Service account token |
| `KUBE_CERTIFICATE` | Base64-encoded CA certificate | | `KUBE_CERTIFICATE` | Base64-encoded CA certificate |
@@ -530,8 +660,8 @@ Per-repo, since different apps may deploy to different clusters.
Created by `kforge secrets apply`. Never appear in Gitea or in `kforge.yml`. Created by `kforge secrets apply`. Never appear in Gitea or in `kforge.yml`.
| Secret name | Contents | | Secret name | Contents |
| --------------------------------- | --------------------------------- | |---|---|
| `{full_name}-db-credentials` | CNPG-managed database credentials | | `{full_name}-db-credentials` | PostgreSQL `username` + alphanumeric `password` |
| `{full_name}-basic-auth` | htpasswd for ingress basic auth | | `{full_name}-basic-auth` | htpasswd for ingress basic auth |
| `{full_name}-cache-credentials` | Valkey/Redis password | | `{full_name}-cache-credentials` | Valkey/Redis password |
| `{full_name}-storage-credentials` | Minio access/secret keys | | `{full_name}-storage-credentials` | Minio access/secret keys |
@@ -549,9 +679,7 @@ kforge.yml ← your app config (committed)
.gitea/ .gitea/
workflows/ workflows/
deploy.yml ← generated by kforge gitea-actions (committed) deploy.yml ← generated by kforge gitea-actions (committed)
.kforge/ preview.yml ← generated by kforge gitea-preview (committed, optional)
secrets.enc.yml ← SOPS-encrypted sensitive config (committed)
kforge.age ← age private key (NEVER committed — goes in Gitea as SOPS_AGE_KEY)
``` ```
Generated manifests (`.kforge-out/`) are never committed — they are created at CI time and discarded after `kubectl apply`. Generated manifests (`.kforge-out/`) are never committed — they are created at CI time and discarded after `kubectl apply`.