From 4ee1b9e13c96121ac61d2c7490c12ec98e967ed0 Mon Sep 17 00:00:00 2001 From: Nathanial Lubitz Date: Mon, 29 Jun 2026 15:14:55 +1000 Subject: [PATCH] more updates --- CLAUDE.md | 167 ++++++++++++ cmd/dns_gitea.go | 160 +++++------ cmd/generate.go | 137 +++++++--- cmd/secrets_apply.go | 97 ++++--- cmd/validate.go | 24 +- example/kforge.yml | 24 +- internal/config/defaults.go | 184 +++++++++---- internal/config/types.go | 127 +++++---- internal/dns/cloudflare.go | 301 --------------------- internal/generator/gitea_actions.go | 206 ++++++++++---- internal/generator/infrastructure.go | 108 ++++++-- internal/generator/manifests.go | 36 +++ kforge.yml | 9 +- pkg/interpolate/interpolate.go | 17 ++ readme.md | 390 ++++++++++++++++++--------- 15 files changed, 1155 insertions(+), 832 deletions(-) create mode 100644 CLAUDE.md delete mode 100644 internal/dns/cloudflare.go diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..364190d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,167 @@ +# kforge + +**kforge** generates production-ready Kubernetes manifests from a single `kforge.yml` in your repository root. Designed for self-hosted homelab deployments on MicroK8s with Gitea Actions CI/CD. + +## Core concept + +``` +kforge.yml → kforge generate → kubectl apply → cluster +``` + +Only `kforge.yml` and generated workflow files are committed. Generated manifests are applied and discarded each CI run. + +## Build and test + +```sh +go build -o kforge . +go test ./... +``` + +## Commands + +| Command | Purpose | +|---|---| +| `kforge validate` | Validate kforge.yml, list required secrets | +| `kforge generate [--env E]` | Generate Kubernetes manifests to `.kforge-out/` | +| `kforge generate --pr-number N` | Generate manifests for a PR preview environment | +| `kforge secrets apply --env E` | Generate and apply random credentials to cluster | +| `kforge secrets apply --pr-number N` | Apply credentials for a PR preview environment | +| `kforge gitea-actions` | Generate `.gitea/workflows/deploy.yml` | +| `kforge gitea-preview` | Generate `.gitea/workflows/preview.yml` for PR previews | + +## Architecture + +### Config system (`internal/config/`) +- `types.go` — all config structs mirroring `kforge.yml` +- `loader.go` — YAML parsing and structural validation +- `defaults.go` — `ApplyDefaults()`, `ResolveEnvironment()`, `SynthesizePreviewEnvironment()` + +### Generator system (`internal/generator/`) +- `manifests.go` — Service, Deployment, Ingress (with external-dns annotations), Certificate, CronJob +- `infrastructure.go` — CNPG Database CR + db-init Job, Valkey, Minio, NATS, Meilisearch +- `gitea_actions.go` — deploy workflow (`GenerateGiteaActions`) + preview workflow (`GeneratePreviewActions`) + +### CLI (`cmd/`) +- Uses [Cobra](https://github.com/spf13/cobra) for subcommands +- `root.go` — shared `loadConfig()` helper +- Each command calls `loadConfig()` then delegates to generators + +### Token interpolation (`pkg/interpolate/`) +- Built-in: `${name}`, `${tenant}`, `${env}`, `${env_prefix}`, `${full_name}`, `${namespace}`, `${image_tag}` +- Falls back to `os.Getenv()` — CI secrets (e.g. `${KFORGE_NODE_IP}`) are injected this way +- `PGIdentifier()` converts a kforge name to a valid unquoted PostgreSQL identifier + +## DNS and TLS + +DNS is managed by **external-dns** running in the cluster — no Cloudflare API calls from kforge. + +When a host has `dns_record: true`, kforge adds to the Ingress: +```yaml +annotations: + external-dns.alpha.kubernetes.io/hostname: "app.example.com" + external-dns.alpha.kubernetes.io/target: "" +``` + +`dns.target` in `kforge.yml` sets the target value (your node IP or a static hostname). Supports `${KFORGE_NODE_IP}` token which is resolved from the CI environment. + +TLS is managed by **cert-manager** via a `ClusterIssuer`. A `Certificate` CR is generated for each host with `tls: true`. No separate DNS step needed in the workflow. + +## CNPG database (centralized cluster) + +For a centralized CNPG cluster, kforge generates: +1. A CNPG `Database` CR — declaratively manages the database lifecycle +2. A `db-init` Kubernetes Job — creates the PostgreSQL role and syncs its password on every deploy + +`kforge secrets apply` creates a `${full_name}-db-credentials` Secret containing an alphanumeric `username` and random `password` **before** the Job runs. The password is alphanumeric-only so it can be safely used in shell commands within the Job. + +Required cluster resources: +- CNPG superuser Secret named by `cluster.cnpg.superuser_secret` (default: `cnpg-main-superuser`) +- CNPG cluster named by `cluster.cnpg.cluster_name` (default: `cnpg-main`) + +## PR Preview environments + +Add a `preview:` block to `kforge.yml` and run `kforge gitea-preview` to generate `.gitea/workflows/preview.yml`. + +The preview workflow: +- **PR open / sync**: Creates namespace `preview-pr-{N}`, builds image tagged `:pr-{N}`, deploys app +- **PR close**: Deletes the namespace (removes all preview resources) + +`kforge generate --pr-number N` and `kforge secrets apply --pr-number N` synthesize the preview environment at runtime using `preview.base_environment` settings with namespace/hostname overrides. + +A `Namespace` manifest is included in the generated output so `kubectl apply` creates it automatically. + +## Required secrets (Gitea) + +### Org-level +- `DOCKER_USERNAME`, `DOCKER_PASSWORD` — registry auth +- `KFORGE_NODE_IP` — cluster node IP (used as external-dns target) + +### Repo-level +- `KUBE_HOST`, `KUBE_TOKEN`, `KUBE_CERTIFICATE` — kubectl auth + +## kforge.yml reference + +```yaml +meta: + name: my-app + tenant: my-tenant + +registry: + url: registry.example.com + pull_secret: regcred # default + +dns: + target: ${KFORGE_NODE_IP} # external-dns annotation target + +cluster: + tls_issuer: letsencrypt-prod # default + ingress_class: nginx # default + cnpg: + host: cnpg-main-rw.default.svc.cluster.local # default + cluster_name: cnpg-main # default + superuser_secret: cnpg-main-superuser # default + +defaults: + port: 3000 + health_check: + path: /healthcheck + resources: + requests: { cpu: 100m, memory: 128Mi } + limits: { cpu: 500m, memory: 512Mi } + +infrastructure: + database: + provider: cnpg + cache: + provider: valkey + mode: standalone + +preview: + enabled: true + base_environment: staging + namespace_prefix: preview-pr # namespace = preview-pr-{N} + hostname_template: "pr-${PR_NUMBER}.${name}.example.com" + +environments: + staging: + namespace: staging + image_tag: latest + ingress: + hosts: + - hostname: app-staging.example.com + tls: true + dns_record: true + + production: + namespace: production + image_tag: latest + ingress: + hosts: + - hostname: app.example.com + tls: true + dns_record: true + infrastructure: + cache: + mode: cluster + replicas: 3 +``` diff --git a/cmd/dns_gitea.go b/cmd/dns_gitea.go index 75a1b7d..97636ef 100644 --- a/cmd/dns_gitea.go +++ b/cmd/dns_gitea.go @@ -4,88 +4,11 @@ import ( "fmt" "os" - "kforge/internal/config" - dnsProvider "kforge/internal/dns" "kforge/internal/generator" "github.com/spf13/cobra" ) -// ------------------------------------------------------------ -// kforge dns ensure -// ------------------------------------------------------------ - -var dnsCmd = &cobra.Command{ - Use: "dns", - Short: "Manage DNS records for kforge environments", -} - -var dnsEnsureEnvs []string - -var dnsEnsureCmd = &cobra.Command{ - Use: "ensure", - Short: "Create or update DNS A records for ingress hosts", - Long: `For each ingress host with dns_record: true, creates or updates -an A record pointing to KFORGE_NODE_IP. - -Idempotent — safe to run on every deploy. Skips hosts where the -record already points to the correct IP. - -Examples: - kforge dns ensure --env staging - kforge dns ensure --env staging --env production`, - RunE: runDNSEnsure, -} - -func init() { - dnsEnsureCmd.Flags().StringArrayVarP(&dnsEnsureEnvs, "env", "e", nil, - "Environment(s) to ensure DNS records for (default: all)") - dnsCmd.AddCommand(dnsEnsureCmd) - rootCmd.AddCommand(dnsCmd) -} - -func runDNSEnsure(cmd *cobra.Command, args []string) error { - cfg, err := loadConfig() - if err != nil { - return err - } - - if cfg.DNS.SkipDNS { - fmt.Println("dns.skip_dns is true — skipping DNS management") - return nil - } - - nodeIP := cfg.DNS.NodeIP - if nodeIP == "" { - nodeIP = os.Getenv("KFORGE_NODE_IP") - } - if nodeIP == "" { - return fmt.Errorf("node IP not set: add dns.node_ip to kforge.yml or set KFORGE_NODE_IP") - } - - provider, err := dnsProvider.NewProvider(cfg.DNS) - if err != nil { - return fmt.Errorf("initialising DNS provider: %w", err) - } - - envKeys := dnsEnsureEnvs - if len(envKeys) == 0 { - envKeys = config.EnvironmentKeys(cfg) - } - - for _, envKey := range envKeys { - fmt.Printf("\nEnsuring DNS records for: %s\n", envKey) - env, err := config.ResolveEnvironment(cfg, envKey) - if err != nil { - return err - } - if err := dnsProvider.EnsureRecordsForEnvironment(provider, &env, nodeIP); err != nil { - return fmt.Errorf("env %q: %w", envKey, err) - } - } - return nil -} - // ------------------------------------------------------------ // kforge gitea-actions // ------------------------------------------------------------ @@ -98,16 +21,17 @@ var ( var giteaActionsCmd = &cobra.Command{ Use: "gitea-actions", - Short: "Generate a Gitea Actions workflow for this app", - Long: `Generates a complete .gitea/workflows/deploy.yml that: - - Builds and pushes the Docker image on every push to main + Short: "Generate a Gitea Actions deploy workflow for this app", + Long: `Generates .gitea/workflows/deploy.yml that: + - Builds and pushes the Docker image on push to main - Runs kforge validate - Applies cluster secrets (idempotent) - - Ensures DNS records - Generates manifests and applies them with kubectl - Rolls out the deployment -The generated workflow replaces your hand-written deploy.yml. +DNS is handled automatically by external-dns reading the Ingress +annotations that kforge writes — no separate DNS step needed. + Re-run whenever you add environments or change deploy options. Examples: @@ -142,29 +66,87 @@ func runGiteaActions(cmd *cobra.Command, args []string) error { return fmt.Errorf("generating workflow: %w", err) } - if giteaActionsOutput == "-" { - fmt.Print(workflow) + return writeWorkflow(giteaActionsOutput, workflow) +} + +// ------------------------------------------------------------ +// kforge gitea-preview +// ------------------------------------------------------------ + +var giteaPreviewOutput string + +var giteaPreviewCmd = &cobra.Command{ + Use: "gitea-preview", + Short: "Generate a Gitea Actions workflow for PR preview environments", + Long: `Generates .gitea/workflows/preview.yml that: + - On PR open/sync: builds a PR-tagged image, applies secrets, + generates manifests (including a Namespace), and deploys. + - On PR close: deletes the preview namespace and all resources. + +Requires preview.enabled: true in kforge.yml. + +Example kforge.yml preview block: + preview: + enabled: true + base_environment: staging + namespace_prefix: preview-pr + hostname_template: "pr-${PR_NUMBER}.${name}.example.com" + +Examples: + kforge gitea-preview + kforge gitea-preview --output .gitea/workflows/preview.yml`, + RunE: runGiteaPreview, +} + +func init() { + giteaPreviewCmd.Flags().StringVarP(&giteaPreviewOutput, "output", "o", + ".gitea/workflows/preview.yml", + "Output path for the generated preview workflow file") + rootCmd.AddCommand(giteaPreviewCmd) +} + +func runGiteaPreview(cmd *cobra.Command, args []string) error { + cfg, err := loadConfig() + if err != nil { + return err + } + + workflow, err := generator.GeneratePreviewActions(cfg) + if err != nil { + return fmt.Errorf("generating preview workflow: %w", err) + } + + return writeWorkflow(giteaPreviewOutput, workflow) +} + +// ------------------------------------------------------------ +// Shared helpers +// ------------------------------------------------------------ + +func writeWorkflow(path, content string) error { + if path == "-" { + fmt.Print(content) return nil } // Ensure parent directory exists. - dir := giteaActionsOutput + dir := path for i := len(dir) - 1; i >= 0; i-- { - if dir[i] == '/' { + if dir[i] == '/' || dir[i] == '\\' { dir = dir[:i] break } } - if dir != giteaActionsOutput { + if dir != path { if err := os.MkdirAll(dir, 0o755); err != nil { return fmt.Errorf("creating output directory: %w", err) } } - if err := os.WriteFile(giteaActionsOutput, []byte(workflow), 0o644); err != nil { + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { return fmt.Errorf("writing workflow: %w", err) } - fmt.Printf("✓ Gitea Actions workflow written to %s\n", giteaActionsOutput) + fmt.Printf("✓ Workflow written to %s\n", path) return nil } diff --git a/cmd/generate.go b/cmd/generate.go index 566f409..bf63aed 100644 --- a/cmd/generate.go +++ b/cmd/generate.go @@ -13,27 +13,32 @@ import ( ) var ( - generateEnvs []string - generateOutput string - generateDry bool + generateEnvs []string + generateOutput string + generateDry bool + generatePRNumber string ) var generateCmd = &cobra.Command{ Use: "generate", Short: "Generate Kubernetes manifests from kforge.yml", - Long: `Reads kforge.yml (or the file specified with --config), resolves -each requested environment, and writes flat Kubernetes manifest -files to the output directory. + Long: `Reads kforge.yml, resolves each requested environment, and writes +flat Kubernetes manifest files to the output directory. If no --env flags are given, manifests are generated for all environments defined in kforge.yml. +Use --pr-number to generate manifests for a PR preview environment. +A Namespace manifest is prepended so kubectl apply creates the +namespace automatically. + Examples: kforge generate kforge generate --env staging kforge generate --env staging --env production kforge generate --env production --output .kube/ - kforge generate --dry-run`, + kforge generate --dry-run + kforge generate --pr-number 42`, RunE: runGenerate, } @@ -44,6 +49,8 @@ func init() { "Directory to write generated manifests into") generateCmd.Flags().BoolVar(&generateDry, "dry-run", false, "Print manifests to stdout instead of writing files") + generateCmd.Flags().StringVar(&generatePRNumber, "pr-number", "", + "PR number — synthesizes and generates a preview environment") rootCmd.AddCommand(generateCmd) } @@ -53,6 +60,15 @@ func runGenerate(cmd *cobra.Command, args []string) error { return err } + // Preview mode: synthesize the preview environment. + if generatePRNumber != "" { + env, err := config.SynthesizePreviewEnvironment(cfg, generatePRNumber) + if err != nil { + return fmt.Errorf("synthesizing preview environment: %w", err) + } + return generateForPreview(cfg, &env, generatePRNumber) + } + envKeys := generateEnvs if len(envKeys) == 0 { envKeys = config.EnvironmentKeys(cfg) @@ -78,30 +94,9 @@ func generateForEnv(cfg *config.KforgeConfig, envKey string) error { return err } - // Core manifests (Service, Deployment, Ingress, Certs). - coreYAML, err := generator.GenerateAll(&env, cfg) + coreYAML, infraManifests, err := buildManifests(cfg, &env) if err != nil { - return fmt.Errorf("generating core manifests: %w", err) - } - - // Infrastructure manifests. - infraManifests, err := generator.GenerateInfrastructure(&env) - if err != nil { - return fmt.Errorf("generating infrastructure manifests: %w", err) - } - - // Collect all infrastructure env vars and append to deployment. - // We re-generate core manifests after injecting infra env vars. - var infraEnvVars []config.EnvVarConfig - for _, m := range infraManifests { - infraEnvVars = append(infraEnvVars, m.EnvVars...) - } - if len(infraEnvVars) > 0 { - env.EnvVars = config.MergeEnvVars(env.EnvVars, infraEnvVars) - coreYAML, err = generator.GenerateAll(&env, cfg) - if err != nil { - return fmt.Errorf("re-generating core manifests with infra vars: %w", err) - } + return err } if generateDry { @@ -112,14 +107,16 @@ func generateForEnv(cfg *config.KforgeConfig, envKey string) error { return nil } - // Write core manifest. + if err := os.MkdirAll(generateOutput, 0o755); err != nil { + return fmt.Errorf("creating output directory: %w", err) + } + coreFile := filepath.Join(generateOutput, envKey+"-core.yaml") if err := os.WriteFile(coreFile, []byte(coreYAML), 0o644); err != nil { return fmt.Errorf("writing core manifest: %w", err) } fmt.Printf(" ✓ %s\n", coreFile) - // Write infra manifests. for _, m := range infraManifests { infraFile := filepath.Join(generateOutput, envKey+"-infra-"+m.Name+".yaml") if err := os.WriteFile(infraFile, []byte(m.Content), 0o644); err != nil { @@ -131,6 +128,82 @@ func generateForEnv(cfg *config.KforgeConfig, envKey string) error { return nil } +// generateForPreview generates manifests for a synthesized preview +// environment, prepending a Namespace manifest so kubectl apply +// creates the namespace in a single pass. +func generateForPreview(cfg *config.KforgeConfig, env *config.ResolvedEnvironment, prNumber string) error { + coreYAML, infraManifests, err := buildManifests(cfg, env) + if err != nil { + return err + } + + // Prepend a Namespace manifest so kubectl apply is self-contained. + nsYAML := generator.Namespace(env.Namespace, map[string]string{ + "managed-by": "kforge", + "kforge/preview": "true", + "kforge/pr": prNumber, + }) + coreYAML = nsYAML + generator.Separator + coreYAML + + envKey := "preview-" + prNumber + + if generateDry { + printDryRun(envKey, "core", coreYAML) + for _, m := range infraManifests { + printDryRun(envKey, m.Name, m.Content) + } + return nil + } + + if err := os.MkdirAll(generateOutput, 0o755); err != nil { + return fmt.Errorf("creating output directory: %w", err) + } + + coreFile := filepath.Join(generateOutput, envKey+"-core.yaml") + if err := os.WriteFile(coreFile, []byte(coreYAML), 0o644); err != nil { + return fmt.Errorf("writing core manifest: %w", err) + } + fmt.Printf(" ✓ %s\n", coreFile) + + for _, m := range infraManifests { + infraFile := filepath.Join(generateOutput, envKey+"-infra-"+m.Name+".yaml") + if err := os.WriteFile(infraFile, []byte(m.Content), 0o644); err != nil { + return fmt.Errorf("writing %s manifest: %w", m.Name, err) + } + fmt.Printf(" ✓ %s\n", infraFile) + } + + return nil +} + +// buildManifests generates core + infra manifests for a resolved environment, +// injecting infra env vars into the deployment. +func buildManifests(cfg *config.KforgeConfig, env *config.ResolvedEnvironment) (string, []generator.InfraManifest, error) { + coreYAML, err := generator.GenerateAll(env, cfg) + if err != nil { + return "", nil, fmt.Errorf("generating core manifests: %w", err) + } + + infraManifests, err := generator.GenerateInfrastructure(env) + if err != nil { + return "", nil, fmt.Errorf("generating infrastructure manifests: %w", err) + } + + var infraEnvVars []config.EnvVarConfig + for _, m := range infraManifests { + infraEnvVars = append(infraEnvVars, m.EnvVars...) + } + if len(infraEnvVars) > 0 { + env.EnvVars = config.MergeEnvVars(env.EnvVars, infraEnvVars) + coreYAML, err = generator.GenerateAll(env, cfg) + if err != nil { + return "", nil, fmt.Errorf("re-generating core manifests with infra vars: %w", err) + } + } + + return coreYAML, infraManifests, nil +} + func printDryRun(envKey, name, content string) { fmt.Printf("\n%s\n# --- %s / %s ---\n%s\n", generator.Separator, diff --git a/cmd/secrets_apply.go b/cmd/secrets_apply.go index 880c3f6..0a4f849 100644 --- a/cmd/secrets_apply.go +++ b/cmd/secrets_apply.go @@ -10,17 +10,20 @@ import ( "strings" "kforge/internal/config" + "kforge/pkg/interpolate" "github.com/spf13/cobra" ) -// passwordChars mirrors the character set from your original -// shell command: A-Za-z0-9 + printable special chars. +// passwordChars is the character set for general-purpose passwords. +// Database passwords use alphanumeric-only (see generateAlphanumeric) +// so they are safe to embed in shell commands inside the db-init Job. const passwordChars = `ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!"#$%&'()*+,-./:;<=>?@[\]^_{|}~` var ( - secretsApplyEnvs []string - secretsApplyForce bool + secretsApplyEnvs []string + secretsApplyForce bool + secretsApplyPRNumber string ) var secretsApplyCmd = &cobra.Command{ @@ -30,24 +33,25 @@ var secretsApplyCmd = &cobra.Command{ services and creates/updates Kubernetes Secrets in the cluster. Secrets are created with kubectl — KUBE_HOST, KUBE_TOKEN, and -KUBE_CERTIFICATE must be set in the environment (Gitea injects -these automatically during CI runs). +KUBE_CERTIFICATE must be set in the environment. Already-existing secrets are NOT overwritten unless --force is passed. This prevents accidental credential rotation. Examples: kforge secrets apply --env staging - kforge secrets apply --env production --force`, + kforge secrets apply --env production --force + kforge secrets apply --pr-number 42`, RunE: runSecretsApply, } func init() { secretsApplyCmd.Flags().StringArrayVarP(&secretsApplyEnvs, "env", "e", nil, - "Environment(s) to apply secrets for (required)") + "Environment(s) to apply secrets for") secretsApplyCmd.Flags().BoolVar(&secretsApplyForce, "force", false, "Overwrite existing secrets (triggers credential rotation)") - _ = secretsApplyCmd.MarkFlagRequired("env") + secretsApplyCmd.Flags().StringVar(&secretsApplyPRNumber, "pr-number", "", + "PR number — synthesizes a preview environment instead of a named env") secretsCmd.AddCommand(secretsApplyCmd) } @@ -57,30 +61,58 @@ func runSecretsApply(cmd *cobra.Command, args []string) error { return err } + // Preview mode: synthesize the preview environment. + if secretsApplyPRNumber != "" { + fmt.Printf("\nApplying secrets for preview PR #%s\n", secretsApplyPRNumber) + env, err := config.SynthesizePreviewEnvironment(cfg, secretsApplyPRNumber) + if err != nil { + return fmt.Errorf("synthesizing preview environment: %w", err) + } + return applySecretsForEnv(cfg, &env) + } + + if len(secretsApplyEnvs) == 0 { + return fmt.Errorf("specify --env or --pr-number") + } + for _, envKey := range secretsApplyEnvs { fmt.Printf("\nApplying secrets for environment: %s\n", envKey) - if err := applySecretsForEnv(cfg, envKey); err != nil { + env, err := config.ResolveEnvironment(cfg, envKey) + if err != nil { + return fmt.Errorf("env %q: %w", envKey, err) + } + if err := applySecretsForEnv(cfg, &env); err != nil { return fmt.Errorf("env %q: %w", envKey, err) } } return nil } -func applySecretsForEnv(cfg *config.KforgeConfig, envKey string) error { - env, err := config.ResolveEnvironment(cfg, envKey) - if err != nil { - return err - } - - // Basic auth htpasswd secret +func applySecretsForEnv(cfg *config.KforgeConfig, env *config.ResolvedEnvironment) error { if env.Ingress.Auth.Enabled { - if err := applyBasicAuthSecret(&env); err != nil { + if err := applyBasicAuthSecret(env); err != nil { return fmt.Errorf("basic auth: %w", err) } } infra := env.Infrastructure + // Database: create credentials Secret before the db-init Job runs. + // Password is alphanumeric-only so it's safe in the Job's shell commands. + if infra.Database != nil { + pgUser := interpolate.PGIdentifier(env.FullName) + if err := applyGenericSecret( + env.FullName+"-db-credentials", + env.Namespace, + map[string]string{ + "username": pgUser, + "password": generateAlphanumeric(32), + }, + ); err != nil { + return fmt.Errorf("db credentials: %w", err) + } + } + if infra.Cache != nil { if err := applyGenericSecret( env.FullName+"-cache-credentials", @@ -143,8 +175,6 @@ func applyBasicAuthSecret(env *config.ResolvedEnvironment) error { for _, username := range auth.Users { password := generatePassword(32) - // Generate bcrypt hash using htpasswd (available on most systems) - // or fall back to a simple SHA1 if htpasswd isn't available. hash, err := generateHTPasswdEntry(username, password) if err != nil { return fmt.Errorf("hashing password for %s: %w", username, err) @@ -164,32 +194,25 @@ func applyBasicAuthSecret(env *config.ResolvedEnvironment) error { } // generateHTPasswdEntry produces a username:bcrypt_hash string. -// Uses htpasswd binary if available, otherwise uses openssl. func generateHTPasswdEntry(username, password string) (string, error) { - // Try htpasswd first (apache2-utils package). if path, err := exec.LookPath("htpasswd"); err == nil { out, err := exec.Command(path, "-nbB", username, password).Output() if err == nil { return strings.TrimSpace(string(out)), nil } } - - // Fall back to openssl passwd -apr1 (MD5 crypt, still widely supported). if path, err := exec.LookPath("openssl"); err == nil { out, err := exec.Command(path, "passwd", "-apr1", password).Output() if err == nil { return username + ":" + strings.TrimSpace(string(out)), nil } } - return "", fmt.Errorf("neither htpasswd nor openssl found; install apache2-utils") } -// applyGenericSecret creates or updates a Kubernetes Secret using -// kubectl. Skips creation if the secret already exists and --force -// was not passed. +// applyGenericSecret creates or updates a Kubernetes Secret using kubectl. +// Skips creation if the secret already exists and --force was not passed. func applyGenericSecret(name, namespace string, data map[string]string) error { - // Check if secret already exists. checkCmd := kubectlCmd("get", "secret", name, "-n", namespace, "--ignore-not-found") out, err := checkCmd.Output() if err != nil { @@ -202,7 +225,6 @@ func applyGenericSecret(name, namespace string, data map[string]string) error { return nil } - // Build kubectl create secret generic args. args := []string{ "create", "secret", "generic", name, "-n", namespace, @@ -214,7 +236,6 @@ func applyGenericSecret(name, namespace string, data map[string]string) error { args = append(args, fmt.Sprintf("--from-literal=%s=%s", k, v)) } - // Pipe through kubectl apply to handle create-or-update. createCmd := kubectlCmd(args...) yamlBytes, err := createCmd.Output() if err != nil { @@ -238,9 +259,8 @@ func applyGenericSecret(name, namespace string, data map[string]string) error { return nil } -// kubectlCmd builds a kubectl invocation using the KUBE_HOST, -// KUBE_TOKEN, and KUBE_CERTIFICATE env vars for auth — the same -// pattern used in your existing Gitea Actions workflow. +// kubectlCmd builds a kubectl invocation using KUBE_HOST, KUBE_TOKEN, +// and KUBE_CERTIFICATE env vars for auth. func kubectlCmd(args ...string) *exec.Cmd { base := []string{} @@ -251,11 +271,8 @@ func kubectlCmd(args ...string) *exec.Cmd { base = append(base, "--token="+token) } if cert := os.Getenv("KUBE_CERTIFICATE"); cert != "" { - // KUBE_CERTIFICATE is the base64-encoded CA cert. - // Decode it to a temp file or pass inline. decoded, err := base64.StdEncoding.DecodeString(cert) if err == nil { - // Write to a temp file for kubectl. f, err := os.CreateTemp("", "kforge-ca-*.crt") if err == nil { _, _ = f.Write(decoded) @@ -264,7 +281,6 @@ func kubectlCmd(args ...string) *exec.Cmd { } } } else { - // No cert provided — use insecure skip (matches your current workflow). base = append(base, "--insecure-skip-tls-verify=true") } @@ -276,9 +292,6 @@ func kubectlCmd(args ...string) *exec.Cmd { // Password generation // ------------------------------------------------------------ -// generatePassword produces a cryptographically random password -// of length n using the full printable ASCII character set. -// Mirrors: tr -dc 'A-Za-z0-9!"#$%&...' = 0 { + env.Image = env.Image[:idx+1] + "pr-" + prNumber + } + + // Override ingress hostname using the template. + hostnameTemplate := cfg.Preview.HostnameTemplate + if hostnameTemplate == "" { + hostnameTemplate = "pr-${PR_NUMBER}.${name}.example.com" + } + previewTokens := interpolate.Tokens{ + "PR_NUMBER": prNumber, + "name": cfg.Meta.Name, + "tenant": cfg.Meta.Tenant, + } + hostname := interpolate.Apply(hostnameTemplate, previewTokens) + + env.Ingress = IngressConfig{ + Hosts: []IngressHost{{ + Hostname: hostname, + TLS: true, + DNSRecord: true, + }}, + Auth: IngressAuth{ + SecretName: env.FullName + "-basic-auth", + }, + } + + return env, nil +} + // ------------------------------------------------------------ // Internal resolution helpers // ------------------------------------------------------------ @@ -350,7 +442,6 @@ func resolveEnvPrefix(envKey string, override *string) string { if override != nil && *override != "" { return *override } - // Default: first 4 chars of envKey, or full key if shorter. if len(envKey) <= 4 { return envKey } @@ -361,13 +452,10 @@ func resolveNamespace(envKey, explicit, pattern string) string { if explicit != "" { return explicit } - // Apply the namespace pattern (simple token replace here; - // full interpolation runs later via the interpolate package). - result := pattern - if result == "" { + if pattern == "" { return envKey } - return result + return pattern } func resolveFullName(cfg *KforgeConfig, prefix string, env EnvironmentConfig) string { @@ -379,7 +467,6 @@ func resolveFullName(cfg *KforgeConfig, prefix string, env EnvironmentConfig) st func resolveRegistry(cfg *KforgeConfig, env EnvironmentConfig) RegistryConfig { base := cfg.Registry - // Resolve the repository token now that we have meta values. if base.Repository == "${tenant}/${name}" || base.Repository == "" { base.Repository = cfg.Meta.Tenant + "/" + cfg.Meta.Name } @@ -428,15 +515,6 @@ func mergeEnvVars(base, override []EnvVarConfig) []EnvVarConfig { // mergeInfrastructure performs a shallow merge of root infra // defaults with per-environment overrides. -// -// Rules: -// - If root has a service block with no explicit enabled:false, -// it is enabled in every environment. -// - An env block with enabled:false disables the service. -// - An env block with partial fields overrides only those fields; -// everything else inherits from root. -// - If root has no block for a service, env can still enable it -// by providing its own block (enabled defaults to true if present). func mergeInfrastructure(root, env *InfrastructureConfig) ResolvedInfrastructure { return ResolvedInfrastructure{ Database: mergeDatabase(root.Database, env.Database), @@ -635,14 +713,12 @@ func resolveCronJobs(jobs []CronJobConfig, deploymentImage string, deploymentEnv for _, job := range jobs { r := ResolvedCronJob{CronJobConfig: job} - // Image if job.ImageOverride != nil && *job.ImageOverride != "" { r.Image = *job.ImageOverride } else { r.Image = deploymentImage } - // Env vars: deployment vars first, then job-specific (with merge) inheritEnv := job.InheritEnv == nil || *job.InheritEnv if inheritEnv { r.EnvVars = mergeEnvVars(deploymentEnvVars, job.EnvVars) @@ -650,12 +726,10 @@ func resolveCronJobs(jobs []CronJobConfig, deploymentImage string, deploymentEnv r.EnvVars = job.EnvVars } - // Resources if job.Resources == nil { r.Resources = &defaultResources } - // Tuning defaults if r.RestartPolicy == "" { r.RestartPolicy = DefaultRestartPolicy } diff --git a/internal/config/types.go b/internal/config/types.go index f9b25ff..a683cb4 100644 --- a/internal/config/types.go +++ b/internal/config/types.go @@ -6,13 +6,14 @@ package config // KforgeConfig is the top-level struct unmarshalled from kforge.yml. type KforgeConfig struct { - Meta MetaConfig `yaml:"meta"` - Registry RegistryConfig `yaml:"registry"` - DNS DNSConfig `yaml:"dns"` - Cluster ClusterConfig `yaml:"cluster"` - Defaults DefaultsConfig `yaml:"defaults"` - Infrastructure InfrastructureConfig `yaml:"infrastructure"` - Environments map[string]EnvironmentConfig `yaml:"environments"` + Meta MetaConfig `yaml:"meta"` + Registry RegistryConfig `yaml:"registry"` + DNS DNSConfig `yaml:"dns"` + Cluster ClusterConfig `yaml:"cluster"` + Defaults DefaultsConfig `yaml:"defaults"` + Infrastructure InfrastructureConfig `yaml:"infrastructure"` + Preview PreviewConfig `yaml:"preview,omitempty"` + Environments map[string]EnvironmentConfig `yaml:"environments"` } // ------------------------------------------------------------ @@ -38,24 +39,21 @@ type RegistryConfig struct { // ------------------------------------------------------------ // DNS +// +// DNS records and TLS are managed inside the cluster: +// - external-dns reads Ingress annotations and creates DNS records +// - cert-manager issues TLS certificates via a ClusterIssuer +// +// kforge emits the appropriate annotations on the Ingress resource +// for each host with dns_record: true. // ------------------------------------------------------------ type DNSConfig struct { - Provider string `yaml:"provider"` // "cloudflare" - Cloudflare CloudflareConfig `yaml:"cloudflare"` - NodeIP string `yaml:"node_ip,omitempty"` // default: ${KFORGE_NODE_IP} - SkipDNS bool `yaml:"skip_dns,omitempty"` -} - -type CloudflareConfig struct { - APIToken string `yaml:"api_token"` - Zones []ZoneEntry `yaml:"zones"` - Proxied bool `yaml:"proxied,omitempty"` -} - -type ZoneEntry struct { - Name string `yaml:"name"` - ZoneID string `yaml:"zone_id"` + // Target is the value written to the external-dns target annotation. + // Typically the cluster node's public IP or a static hostname. + // Supports token interpolation (e.g. ${KFORGE_NODE_IP}). + Target string `yaml:"target,omitempty"` + SkipDNS bool `yaml:"skip_dns,omitempty"` // true = no external-dns annotations } // ------------------------------------------------------------ @@ -63,15 +61,16 @@ type ZoneEntry struct { // ------------------------------------------------------------ type ClusterConfig struct { - TLSIssuer string `yaml:"tls_issuer,omitempty"` // default: letsencrypt-prod - IngressClass string `yaml:"ingress_class,omitempty"` // default: nginx + TLSIssuer string `yaml:"tls_issuer,omitempty"` // default: letsencrypt-prod + IngressClass string `yaml:"ingress_class,omitempty"` // default: nginx CNPG CNPGConfig `yaml:"cnpg"` NamespacePattern string `yaml:"namespace_pattern,omitempty"` // default: "${env}" } type CNPGConfig struct { - Host string `yaml:"host,omitempty"` // default: cnpg-main-rw.default.svc.cluster.local - HostOverride *string `yaml:"host_override,omitempty"` + Host string `yaml:"host,omitempty"` // default: cnpg-main-rw.default.svc.cluster.local + ClusterName string `yaml:"cluster_name,omitempty"` // default: cnpg-main + SuperuserSecret string `yaml:"superuser_secret,omitempty"` // default: cnpg-main-superuser } // ------------------------------------------------------------ @@ -79,19 +78,19 @@ type CNPGConfig struct { // ------------------------------------------------------------ type DefaultsConfig struct { - ImagePullPolicy string `yaml:"image_pull_policy,omitempty"` // default: Always - Replicas *int `yaml:"replicas,omitempty"` // default: 1 - Port *int `yaml:"port,omitempty"` // default: 3000 - ServiceType string `yaml:"service_type,omitempty"` // default: ClusterIP - Dockerfile string `yaml:"dockerfile,omitempty"` // default: Dockerfile + ImagePullPolicy string `yaml:"image_pull_policy,omitempty"` // default: Always + Replicas *int `yaml:"replicas,omitempty"` // default: 1 + Port *int `yaml:"port,omitempty"` // default: 3000 + ServiceType string `yaml:"service_type,omitempty"` // default: ClusterIP + Dockerfile string `yaml:"dockerfile,omitempty"` // default: Dockerfile HealthCheck HealthCheckConfig `yaml:"health_check"` - Resources ResourceConfig `yaml:"resources"` - EnvVars []EnvVarConfig `yaml:"env_vars,omitempty"` + Resources ResourceConfig `yaml:"resources"` + EnvVars []EnvVarConfig `yaml:"env_vars,omitempty"` } type HealthCheckConfig struct { - Path string `yaml:"path,omitempty"` // default: /healthcheck - Port *int `yaml:"port,omitempty"` // default: defaults.port + Path string `yaml:"path,omitempty"` // default: /healthcheck + Port *int `yaml:"port,omitempty"` // default: defaults.port InitialDelaySeconds int `yaml:"initial_delay_seconds,omitempty"` // default: 15 PeriodSeconds int `yaml:"period_seconds,omitempty"` // default: 10 TimeoutSeconds int `yaml:"timeout_seconds,omitempty"` // default: 5 @@ -155,9 +154,6 @@ type InfrastructureConfig struct { // InfraBase holds the common enabled flag present on every // infrastructure service. Embedded in each service config. type InfraBase struct { - // Enabled defaults to true if the block exists in the root - // infrastructure section, and inherits that value in envs. - // Set explicitly to false to disable for a specific env. Enabled *bool `yaml:"enabled,omitempty"` } @@ -197,6 +193,31 @@ type MonitoringInfraConfig struct { MetricsPort *int `yaml:"metrics_port,omitempty"` // default: defaults.port } +// ------------------------------------------------------------ +// Preview environments +// ------------------------------------------------------------ + +// PreviewConfig defines how PR preview environments are deployed. +// When enabled, `kforge gitea-preview` generates a workflow that +// deploys an isolated environment per pull request. +type PreviewConfig struct { + // Enabled must be true for kforge gitea-preview to generate a workflow. + Enabled bool `yaml:"enabled,omitempty"` + + // BaseEnvironment is the environment whose infrastructure and env var + // settings are inherited (e.g. "staging"). Required when enabled. + BaseEnvironment string `yaml:"base_environment,omitempty"` + + // NamespacePrefix is prepended to the PR number to form the namespace. + // Default: "preview-pr" → namespace "preview-pr-123". + NamespacePrefix string `yaml:"namespace_prefix,omitempty"` + + // HostnameTemplate is the ingress hostname for the preview, using + // ${PR_NUMBER}, ${name}, and ${tenant} as substitution tokens. + // Example: "pr-${PR_NUMBER}.${name}.example.com" + HostnameTemplate string `yaml:"hostname_template,omitempty"` +} + // ------------------------------------------------------------ // Environment // ------------------------------------------------------------ @@ -215,10 +236,10 @@ type EnvironmentConfig struct { EnvVars []EnvVarConfig `yaml:"env_vars,omitempty"` - Ingress IngressConfig `yaml:"ingress"` + Ingress IngressConfig `yaml:"ingress"` Infrastructure InfrastructureConfig `yaml:"infrastructure"` - CronJobs []CronJobConfig `yaml:"cron_jobs,omitempty"` - Lifecycle LifecycleConfig `yaml:"lifecycle"` + CronJobs []CronJobConfig `yaml:"cron_jobs,omitempty"` + Lifecycle LifecycleConfig `yaml:"lifecycle"` } // ------------------------------------------------------------ @@ -233,7 +254,7 @@ type IngressConfig struct { type IngressHost struct { Hostname string `yaml:"hostname"` TLS bool `yaml:"tls"` - DNSRecord bool `yaml:"dns_record,omitempty"` + DNSRecord bool `yaml:"dns_record,omitempty"` // emit external-dns annotations } type IngressAuth struct { @@ -247,19 +268,19 @@ type IngressAuth struct { // ------------------------------------------------------------ type CronJobConfig struct { - Name string `yaml:"name"` - Schedule string `yaml:"schedule"` - Command []string `yaml:"command"` - ImageOverride *string `yaml:"image_override,omitempty"` - InheritEnv *bool `yaml:"inherit_env,omitempty"` // default: true - EnvVars []EnvVarConfig `yaml:"env_vars,omitempty"` + Name string `yaml:"name"` + Schedule string `yaml:"schedule"` + Command []string `yaml:"command"` + ImageOverride *string `yaml:"image_override,omitempty"` + InheritEnv *bool `yaml:"inherit_env,omitempty"` // default: true + EnvVars []EnvVarConfig `yaml:"env_vars,omitempty"` Resources *ResourceConfig `yaml:"resources,omitempty"` // inherits defaults if nil // Kubernetes CronJob tuning - RestartPolicy string `yaml:"restart_policy,omitempty"` // default: OnFailure - ConcurrencyPolicy string `yaml:"concurrency_policy,omitempty"` // default: Forbid - SuccessfulJobsHistoryLimit *int `yaml:"successful_jobs_history,omitempty"` // default: 3 - FailedJobsHistoryLimit *int `yaml:"failed_jobs_history,omitempty"` // default: 1 + RestartPolicy string `yaml:"restart_policy,omitempty"` // default: OnFailure + ConcurrencyPolicy string `yaml:"concurrency_policy,omitempty"` // default: Forbid + SuccessfulJobsHistoryLimit *int `yaml:"successful_jobs_history,omitempty"` // default: 3 + FailedJobsHistoryLimit *int `yaml:"failed_jobs_history,omitempty"` // default: 1 } // ------------------------------------------------------------ @@ -267,8 +288,6 @@ type CronJobConfig struct { // ------------------------------------------------------------ type LifecycleConfig struct { - // If false (default), kforge renames resources when meta.name changes. - // If true, kforge deletes old resources after delete_grace_seconds. Delete bool `yaml:"delete,omitempty"` DeleteGraceSeconds int `yaml:"delete_grace_seconds,omitempty"` // default: 300 } diff --git a/internal/dns/cloudflare.go b/internal/dns/cloudflare.go deleted file mode 100644 index f51078b..0000000 --- a/internal/dns/cloudflare.go +++ /dev/null @@ -1,301 +0,0 @@ -// Package dns provides a provider-agnostic interface for DNS -// record management, with a Cloudflare implementation. -// -// Adding a new provider (Route53, Porkbun, etc.): -// 1. Implement the Provider interface below. -// 2. Add a case in NewProvider(). -// 3. The rest of kforge uses Provider — no other changes needed. -package dns - -import ( - "encoding/json" - "fmt" - "io" - "net/http" - "strings" - "time" - - "kforge/internal/config" -) - -// ------------------------------------------------------------ -// Provider interface -// ------------------------------------------------------------ - -// Provider is the DNS provider contract. Implementations must -// be idempotent — calling EnsureARecord twice with the same -// inputs must not error or create duplicates. -type Provider interface { - // EnsureARecord creates an A record for hostname pointing to - // ip if one does not already exist. If a record exists with a - // different IP, it is updated. No-ops if already correct. - EnsureARecord(hostname, ip string) error - - // DeleteARecord removes the A record for hostname if it exists. - // No-ops if it does not exist. - DeleteARecord(hostname string) error -} - -// NewProvider returns the configured DNS provider. -func NewProvider(cfg config.DNSConfig) (Provider, error) { - switch cfg.Provider { - case "cloudflare": - return newCloudflareProvider(cfg) - case "": - return &noopProvider{}, nil - default: - return nil, fmt.Errorf("unknown dns provider %q (supported: cloudflare)", cfg.Provider) - } -} - -// noopProvider satisfies the interface when DNS management is -// disabled (skip_dns: true or no provider configured). -type noopProvider struct{} - -func (n *noopProvider) EnsureARecord(hostname, ip string) error { return nil } -func (n *noopProvider) DeleteARecord(hostname string) error { return nil } - -// ------------------------------------------------------------ -// Cloudflare implementation -// ------------------------------------------------------------ - -const cfAPIBase = "https://api.cloudflare.com/client/v4" - -type cloudflareProvider struct { - apiToken string - zones []config.ZoneEntry // sorted longest-first for matching - proxied bool - client *http.Client -} - -func newCloudflareProvider(cfg config.DNSConfig) (*cloudflareProvider, error) { - if cfg.Cloudflare.APIToken == "" { - return nil, fmt.Errorf("cloudflare.api_token is required") - } - if len(cfg.Cloudflare.Zones) == 0 { - return nil, fmt.Errorf("cloudflare.zones must have at least one entry") - } - return &cloudflareProvider{ - apiToken: cfg.Cloudflare.APIToken, - zones: cfg.Cloudflare.Zones, - proxied: cfg.Cloudflare.Proxied, - client: &http.Client{Timeout: 15 * time.Second}, - }, nil -} - -// zoneForHostname finds the zone whose name is the longest suffix -// of hostname. This handles both "app.example.com" → "example.com" -// and "app.sub.example.co.uk" → "example.co.uk" if that zone exists. -func (c *cloudflareProvider) zoneForHostname(hostname string) (config.ZoneEntry, error) { - var best config.ZoneEntry - bestLen := 0 - for _, z := range c.zones { - if strings.HasSuffix(hostname, z.Name) && len(z.Name) > bestLen { - best = z - bestLen = len(z.Name) - } - } - if bestLen == 0 { - return config.ZoneEntry{}, fmt.Errorf("no configured zone matches hostname %q", hostname) - } - return best, nil -} - -// EnsureARecord is idempotent: creates if absent, updates if IP -// differs, no-ops if already correct. -func (c *cloudflareProvider) EnsureARecord(hostname, ip string) error { - zone, err := c.zoneForHostname(hostname) - if err != nil { - return err - } - - existing, err := c.getRecord(zone.ZoneID, hostname, "A") - if err != nil { - return fmt.Errorf("checking existing record: %w", err) - } - - if existing != nil { - if existing.Content == ip { - fmt.Printf(" dns: A record %s → %s already correct, skipping\n", hostname, ip) - return nil - } - fmt.Printf(" dns: updating A record %s → %s (was %s)\n", hostname, ip, existing.Content) - return c.updateRecord(zone.ZoneID, existing.ID, hostname, ip) - } - - fmt.Printf(" dns: creating A record %s → %s\n", hostname, ip) - return c.createRecord(zone.ZoneID, hostname, ip) -} - -// DeleteARecord removes the A record for hostname if it exists. -func (c *cloudflareProvider) DeleteARecord(hostname string) error { - zone, err := c.zoneForHostname(hostname) - if err != nil { - return err - } - - existing, err := c.getRecord(zone.ZoneID, hostname, "A") - if err != nil { - return fmt.Errorf("checking existing record: %w", err) - } - if existing == nil { - fmt.Printf(" dns: A record %s not found, skipping delete\n", hostname) - return nil - } - - fmt.Printf(" dns: deleting A record %s\n", hostname) - return c.deleteRecord(zone.ZoneID, existing.ID) -} - -// ------------------------------------------------------------ -// Cloudflare API helpers -// ------------------------------------------------------------ - -type cfRecord struct { - ID string `json:"id"` - Name string `json:"name"` - Type string `json:"type"` - Content string `json:"content"` - Proxied bool `json:"proxied"` - TTL int `json:"ttl"` -} - -type cfListResponse struct { - Success bool `json:"success"` - Errors []cfError `json:"errors"` - Result []cfRecord `json:"result"` -} - -type cfSingleResponse struct { - Success bool `json:"success"` - Errors []cfError `json:"errors"` - Result cfRecord `json:"result"` -} - -type cfError struct { - Code int `json:"code"` - Message string `json:"message"` -} - -func (e cfError) Error() string { - return fmt.Sprintf("CF %d: %s", e.Code, e.Message) -} - -func (c *cloudflareProvider) getRecord(zoneID, name, recType string) (*cfRecord, error) { - url := fmt.Sprintf("%s/zones/%s/dns_records?type=%s&name=%s", cfAPIBase, zoneID, recType, name) - resp, err := c.do("GET", url, nil) - if err != nil { - return nil, err - } - defer resp.Body.Close() - - var list cfListResponse - if err := json.NewDecoder(resp.Body).Decode(&list); err != nil { - return nil, fmt.Errorf("decoding response: %w", err) - } - if !list.Success { - return nil, cfErrors(list.Errors) - } - if len(list.Result) == 0 { - return nil, nil - } - return &list.Result[0], nil -} - -func (c *cloudflareProvider) createRecord(zoneID, name, ip string) error { - body := fmt.Sprintf(`{"type":"A","name":%q,"content":%q,"ttl":1,"proxied":%v}`, - name, ip, c.proxied) - url := fmt.Sprintf("%s/zones/%s/dns_records", cfAPIBase, zoneID) - resp, err := c.do("POST", url, strings.NewReader(body)) - if err != nil { - return err - } - defer resp.Body.Close() - - var result cfSingleResponse - if err := json.NewDecoder(resp.Body).Decode(&result); err != nil { - return fmt.Errorf("decoding response: %w", err) - } - if !result.Success { - return cfErrors(result.Errors) - } - return nil -} - -func (c *cloudflareProvider) updateRecord(zoneID, recordID, name, ip string) error { - body := fmt.Sprintf(`{"type":"A","name":%q,"content":%q,"ttl":1,"proxied":%v}`, - name, ip, c.proxied) - url := fmt.Sprintf("%s/zones/%s/dns_records/%s", cfAPIBase, zoneID, recordID) - resp, err := c.do("PUT", url, strings.NewReader(body)) - if err != nil { - return err - } - defer resp.Body.Close() - - var result cfSingleResponse - if err := json.NewDecoder(resp.Body).Decode(&result); err != nil { - return fmt.Errorf("decoding response: %w", err) - } - if !result.Success { - return cfErrors(result.Errors) - } - return nil -} - -func (c *cloudflareProvider) deleteRecord(zoneID, recordID string) error { - url := fmt.Sprintf("%s/zones/%s/dns_records/%s", cfAPIBase, zoneID, recordID) - resp, err := c.do("DELETE", url, nil) - if err != nil { - return err - } - defer resp.Body.Close() - // Cloudflare returns {"result":{"id":"..."}} on success — we - // don't need to parse it, just check for HTTP errors. - body, _ := io.ReadAll(resp.Body) - if resp.StatusCode >= 400 { - return fmt.Errorf("delete failed (%d): %s", resp.StatusCode, string(body)) - } - return nil -} - -func (c *cloudflareProvider) do(method, url string, body io.Reader) (*http.Response, error) { - req, err := http.NewRequest(method, url, body) - if err != nil { - return nil, err - } - req.Header.Set("Authorization", "Bearer "+c.apiToken) - req.Header.Set("Content-Type", "application/json") - - resp, err := c.client.Do(req) - if err != nil { - return nil, fmt.Errorf("cloudflare API %s %s: %w", method, url, err) - } - return resp, nil -} - -func cfErrors(errs []cfError) error { - msgs := make([]string, len(errs)) - for i, e := range errs { - msgs[i] = e.Error() - } - return fmt.Errorf("cloudflare API errors: %s", strings.Join(msgs, "; ")) -} - -// ------------------------------------------------------------ -// EnsureRecordsForEnvironment — high-level helper used by -// the apply command and generate pipeline. -// ------------------------------------------------------------ - -// EnsureRecordsForEnvironment creates A records for all ingress -// hosts in the environment that have dns_record: true. -func EnsureRecordsForEnvironment(provider Provider, env *config.ResolvedEnvironment, nodeIP string) error { - for _, host := range env.Ingress.Hosts { - if !host.DNSRecord { - continue - } - if err := provider.EnsureARecord(host.Hostname, nodeIP); err != nil { - return fmt.Errorf("ensuring A record for %s: %w", host.Hostname, err) - } - } - return nil -} diff --git a/internal/generator/gitea_actions.go b/internal/generator/gitea_actions.go index 65c09d1..1b53e6e 100644 --- a/internal/generator/gitea_actions.go +++ b/internal/generator/gitea_actions.go @@ -23,9 +23,8 @@ type GiteaActionsOptions struct { // that builds the Docker image and deploys to each environment // using kforge generate + kubectl apply. // -// The generated file is designed to replace your existing -// hand-written workflow. It assumes kforge is available in the -// PATH (either pre-installed on the runner or fetched as a step). +// DNS is handled by external-dns via Ingress annotations — no +// separate DNS step is needed in the workflow. func GenerateGiteaActions(cfg *config.KforgeConfig, opts GiteaActionsOptions) (string, error) { if opts.Branch == "" { opts.Branch = "main" @@ -39,20 +38,19 @@ func GenerateGiteaActions(cfg *config.KforgeConfig, opts GiteaActionsOptions) (s var b strings.Builder - writeGiteaHeader(&b, cfg, opts) - writeGiteaJobs(&b, cfg, opts) + writeDeployHeader(&b, cfg, opts) + writeDeployJobs(&b, cfg, opts) return b.String(), nil } -func writeGiteaHeader(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) { +func writeDeployHeader(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) { fmt.Fprintf(b, "# Generated by kforge — do not edit manually.\n") fmt.Fprintf(b, "# Re-generate: kforge gitea-actions > .gitea/workflows/deploy.yml\n") fmt.Fprintf(b, "#\n") fmt.Fprintf(b, "# Required Gitea org secrets:\n") - fmt.Fprintf(b, "# DOCKER_USERNAME, DOCKER_PASSWORD, KFORGE_NODE_IP\n") - fmt.Fprintf(b, "# CLOUDFLARE_API_TOKEN, CF_ZONE_ID_* (per zone)\n") - fmt.Fprintf(b, "# SOPS_AGE_KEY\n") + fmt.Fprintf(b, "# DOCKER_USERNAME, DOCKER_PASSWORD\n") + fmt.Fprintf(b, "# KFORGE_NODE_IP (external-dns annotation target)\n") fmt.Fprintf(b, "# Required Gitea repo secrets:\n") fmt.Fprintf(b, "# KUBE_HOST, KUBE_TOKEN, KUBE_CERTIFICATE\n") fmt.Fprintf(b, "\n") @@ -65,30 +63,21 @@ func writeGiteaHeader(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaAc fmt.Fprintf(b, "\n") } -func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) { +func writeDeployJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActionsOptions) { fmt.Fprintf(b, "jobs:\n") fmt.Fprintf(b, " build-and-deploy:\n") fmt.Fprintf(b, " runs-on: ubuntu-latest\n") fmt.Fprintf(b, " steps:\n") - // Checkout writeStep(b, "Checkout", map[string]any{ "uses": "actions/checkout@v4", "with": map[string]any{"fetch-depth": 0}, }) - // Node (optional — only if package.json exists) - writeStep(b, "Setup Node", map[string]any{ - "uses": "actions/setup-node@v4", - "with": map[string]any{"node-version": opts.NodeVersion}, - }) - - // Short SHA writeStep(b, "Create short commit hash", map[string]any{ "run": `echo "SHORT_SHA=$(git rev-parse --short HEAD)" >> $GITHUB_ENV`, }) - // Docker login writeStep(b, "Login to registry", map[string]any{ "uses": "docker/login-action@v2", "with": map[string]any{ @@ -98,7 +87,6 @@ func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActi }, }) - // Docker build + push fullRepo := cfg.Registry.URL + "/" + cfg.Meta.Tenant + "/" + cfg.Meta.Name writeStep(b, "Build and push image", map[string]any{ "uses": "docker/build-push-action@v5", @@ -113,12 +101,10 @@ func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActi }, }) - // Install kforge on runner writeStep(b, "Install kforge", map[string]any{ "run": "KFORGE_VERSION=\"latest\"\ncurl -fsSL \"https://kforge/releases/download/${KFORGE_VERSION}/kforge-linux-amd64\" -o /usr/local/bin/kforge\nchmod +x /usr/local/bin/kforge", }) - // Per-environment deploy steps for _, envKey := range opts.Environments { env, err := config.ResolveEnvironment(cfg, envKey) if err != nil { @@ -129,45 +115,26 @@ func writeGiteaJobs(b *strings.Builder, cfg *config.KforgeConfig, opts GiteaActi } func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *config.ResolvedEnvironment, envKey string) { - label := strings.Title(envKey) //nolint:staticcheck // simple capitalisation + label := strings.Title(envKey) //nolint:staticcheck - // Validate kforge config before doing anything destructive. writeStep(b, fmt.Sprintf("Validate kforge config (%s)", label), map[string]any{ "run": "kforge validate", - "env": giteaSecretEnv(), + "env": giteaNodeIPEnv(), }) - // Apply cluster secrets (only creates if missing — idempotent). writeStep(b, fmt.Sprintf("Apply cluster secrets (%s)", label), map[string]any{ "run": fmt.Sprintf("kforge secrets apply --env %s", envKey), "env": giteaKubeEnv(), }) - // DNS records - hasDNSHosts := false - for _, h := range env.Ingress.Hosts { - if h.DNSRecord { - hasDNSHosts = true - break - } - } - if hasDNSHosts && !cfg.DNS.SkipDNS { - writeStep(b, fmt.Sprintf("Ensure DNS records (%s)", label), map[string]any{ - "run": fmt.Sprintf("kforge dns ensure --env %s", envKey), - "env": mergeMaps(giteaSecretEnv(), giteaKubeEnv()), - }) - } - - // Generate manifests writeStep(b, fmt.Sprintf("Generate manifests (%s)", label), map[string]any{ "run": fmt.Sprintf( "kforge generate --env %s --output .kforge-out --set image_tag=${{ env.SHORT_SHA }}", envKey, ), - "env": giteaSecretEnv(), + "env": giteaNodeIPEnv(), }) - // kubectl apply writeStep(b, fmt.Sprintf("Apply manifests (%s)", label), map[string]any{ "uses": "actions-hub/kubectl@master", "env": giteaKubeEnv(), @@ -179,7 +146,6 @@ func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *conf }, }) - // Apply infra manifests if any infrastructure is enabled infra := env.Infrastructure if infra.Database != nil || infra.Cache != nil || infra.Storage != nil || infra.Queue != nil || infra.Search != nil || infra.Monitoring != nil { @@ -195,7 +161,6 @@ func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *conf }) } - // Rollout restart writeStep(b, fmt.Sprintf("Rollout restart (%s)", label), map[string]any{ "uses": "actions-hub/kubectl@master", "env": giteaKubeEnv(), @@ -208,11 +173,143 @@ func writeEnvDeploySteps(b *strings.Builder, cfg *config.KforgeConfig, env *conf }) } +// ------------------------------------------------------------ +// Preview workflow +// ------------------------------------------------------------ + +// GeneratePreviewActions produces a Gitea Actions workflow YAML +// that deploys an ephemeral environment per pull request. +// +// On PR open/sync: builds a PR-tagged image, applies secrets, +// generates manifests (including Namespace), and deploys. +// On PR close: deletes the preview namespace, removing all resources. +func GeneratePreviewActions(cfg *config.KforgeConfig) (string, error) { + if !cfg.Preview.Enabled { + return "", fmt.Errorf("preview is not enabled in kforge.yml (set preview.enabled: true)") + } + + nsPrefix := cfg.Preview.NamespacePrefix + if nsPrefix == "" { + nsPrefix = config.DefaultPreviewNamespacePrefix + } + + fullRepo := cfg.Registry.URL + "/" + cfg.Meta.Tenant + "/" + cfg.Meta.Name + + var b strings.Builder + + fmt.Fprintf(&b, "# Generated by kforge — do not edit manually.\n") + fmt.Fprintf(&b, "# Re-generate: kforge gitea-preview > .gitea/workflows/preview.yml\n") + fmt.Fprintf(&b, "#\n") + fmt.Fprintf(&b, "# Required secrets (same as deploy workflow):\n") + fmt.Fprintf(&b, "# DOCKER_USERNAME, DOCKER_PASSWORD\n") + fmt.Fprintf(&b, "# KFORGE_NODE_IP, KUBE_HOST, KUBE_TOKEN, KUBE_CERTIFICATE\n") + fmt.Fprintf(&b, "\n") + fmt.Fprintf(&b, "name: Preview Environment\n") + fmt.Fprintf(&b, "\n") + fmt.Fprintf(&b, "on:\n") + fmt.Fprintf(&b, " pull_request:\n") + fmt.Fprintf(&b, " types: [opened, synchronize, reopened, closed]\n") + fmt.Fprintf(&b, "\n") + fmt.Fprintf(&b, "jobs:\n") + fmt.Fprintf(&b, " preview:\n") + fmt.Fprintf(&b, " runs-on: ubuntu-latest\n") + fmt.Fprintf(&b, " steps:\n") + + writeStep(&b, "Checkout", map[string]any{ + "uses": "actions/checkout@v4", + "with": map[string]any{"fetch-depth": 0}, + }) + + writeStep(&b, "Login to registry", map[string]any{ + "if": "${{ github.event.action != 'closed' }}", + "uses": "docker/login-action@v2", + "with": map[string]any{ + "registry": cfg.Registry.URL, + "username": "${{ secrets.DOCKER_USERNAME }}", + "password": "${{ secrets.DOCKER_PASSWORD }}", + }, + }) + + writeStep(&b, "Build and push preview image", map[string]any{ + "if": "${{ github.event.action != 'closed' }}", + "uses": "docker/build-push-action@v5", + "with": map[string]any{ + "context": ".", + "platforms": "linux/amd64", + "file": cfg.Defaults.Dockerfile, + "push": true, + "tags": fmt.Sprintf("%s:pr-${{ github.event.number }}", fullRepo), + "provenance": false, + "sbom": false, + }, + }) + + writeStep(&b, "Install kforge", map[string]any{ + "if": "${{ github.event.action != 'closed' }}", + "run": "KFORGE_VERSION=\"latest\"\ncurl -fsSL \"https://kforge/releases/download/${KFORGE_VERSION}/kforge-linux-amd64\" -o /usr/local/bin/kforge\nchmod +x /usr/local/bin/kforge", + }) + + writeStep(&b, "Apply preview secrets", map[string]any{ + "if": "${{ github.event.action != 'closed' }}", + "run": "kforge secrets apply --pr-number ${{ github.event.number }}", + "env": giteaKubeEnv(), + }) + + writeStep(&b, "Generate preview manifests", map[string]any{ + "if": "${{ github.event.action != 'closed' }}", + "run": "kforge generate --pr-number ${{ github.event.number }} --output .kforge-out", + "env": map[string]any{ + "KFORGE_NODE_IP": "${{ secrets.KFORGE_NODE_IP }}", + "PR_NUMBER": "${{ github.event.number }}", + }, + }) + + writeStep(&b, "Apply preview manifests", map[string]any{ + "if": "${{ github.event.action != 'closed' }}", + "uses": "actions-hub/kubectl@master", + "env": giteaKubeEnv(), + "with": map[string]any{ + "args": fmt.Sprintf( + "apply -f .kforge-out/ --insecure-skip-tls-verify", + ), + }, + }) + + writeStep(&b, "Wait for preview rollout", map[string]any{ + "if": "${{ github.event.action != 'closed' }}", + "uses": "actions-hub/kubectl@master", + "env": giteaKubeEnv(), + "with": map[string]any{ + "args": fmt.Sprintf( + "rollout status deployment -n %s-${{ github.event.number }} --timeout=120s --insecure-skip-tls-verify", + nsPrefix, + ), + }, + }) + + writeStep(&b, "Destroy preview namespace", map[string]any{ + "if": "${{ github.event.action == 'closed' }}", + "uses": "actions-hub/kubectl@master", + "env": giteaKubeEnv(), + "with": map[string]any{ + "args": fmt.Sprintf( + "delete namespace %s-${{ github.event.number }} --ignore-not-found --insecure-skip-tls-verify", + nsPrefix, + ), + }, + }) + + return b.String(), nil +} + +// ------------------------------------------------------------ +// Shared step helpers +// ------------------------------------------------------------ + // writeStep writes a single step in the jobs.steps list. func writeStep(b *strings.Builder, name string, fields map[string]any) { fmt.Fprintf(b, "\n - name: %s\n", name) - // Emit fields in a stable order. - order := []string{"uses", "run", "with", "env"} + order := []string{"if", "uses", "run", "with", "env"} for _, k := range order { v, ok := fields[k] if !ok { @@ -240,7 +337,6 @@ func writeStep(b *strings.Builder, name string, fields map[string]any) { } func writeMapFields(b *strings.Builder, m map[string]any, indent string) { - // Sort keys for stable output. keys := make([]string, 0, len(m)) for k := range m { keys = append(keys, k) @@ -270,13 +366,11 @@ func writeMapFields(b *strings.Builder, m map[string]any, indent string) { } } -// giteaSecretEnv returns the env block referencing Gitea secrets -// needed for kforge itself (DNS, registry tokens, etc.). -func giteaSecretEnv() map[string]any { +// giteaNodeIPEnv returns env vars needed for kforge commands that +// resolve ${KFORGE_NODE_IP} tokens (validate, generate). +func giteaNodeIPEnv() map[string]any { return map[string]any{ - "CLOUDFLARE_API_TOKEN": "${{ secrets.CLOUDFLARE_API_TOKEN }}", - "KFORGE_NODE_IP": "${{ secrets.KFORGE_NODE_IP }}", - "SOPS_AGE_KEY": "${{ secrets.SOPS_AGE_KEY }}", + "KFORGE_NODE_IP": "${{ secrets.KFORGE_NODE_IP }}", } } @@ -301,8 +395,6 @@ func mergeMaps(maps ...map[string]any) map[string]any { func sortedEnvKeys(cfg *config.KforgeConfig) []string { keys := config.EnvironmentKeys(cfg) - // Put staging/dev before production — a simple heuristic that - // matches the most common deploy order. priority := map[string]int{"dev": 0, "development": 0, "staging": 1, "production": 2, "prod": 2} sort.Slice(keys, func(i, j int) bool { pi, pj := priority[keys[i]], priority[keys[j]] diff --git a/internal/generator/infrastructure.go b/internal/generator/infrastructure.go index bbe3c96..1d02b2c 100644 --- a/internal/generator/infrastructure.go +++ b/internal/generator/infrastructure.go @@ -2,7 +2,6 @@ package generator import ( "fmt" - "strings" "kforge/internal/config" "kforge/pkg/interpolate" @@ -79,18 +78,22 @@ type InfraManifest struct { } // ------------------------------------------------------------ -// Database — CNPG +// Database — CNPG (centralized cluster) // ------------------------------------------------------------ func generateDatabase(env *config.ResolvedEnvironment, db *config.DatabaseInfraConfig) ([]InfraManifest, error) { dbName := db.DatabaseName if dbName == "" { - dbName = interpolate.Slug(env.FullName) + dbName = interpolate.PGIdentifier(env.FullName) } - roleName := dbName + "_role" + // pgUser must be a valid unquoted PostgreSQL identifier; same as + // the username stored in the db-credentials Secret by `kforge secrets apply`. + pgUser := interpolate.PGIdentifier(env.FullName) secretName := env.FullName + "-db-credentials" + cnpgHost := env.CNPGHost - // CNPG Database CR + // CNPG Database CR — declaratively manages the database lifecycle. + // The owner role is created by the db-init Job below. dbManifest := fmt.Sprintf(`apiVersion: postgresql.cnpg.io/v1 kind: Database metadata: @@ -103,47 +106,96 @@ spec: name: %s owner: %s cluster: - name: cnpg-main -`, dbName, env.Namespace, env.FullName, dbName, roleName) + name: %s +`, dbName, env.Namespace, env.FullName, dbName, pgUser, env.CNPGClusterName) - // CNPG Role CR — CNPG creates and rotates the password, - // storing it in the secret named below. - roleManifest := fmt.Sprintf(`apiVersion: postgresql.cnpg.io/v1 -kind: DatabaseRole + // db-init Job — runs on every deploy to ensure the PostgreSQL role + // exists and its password matches the Secret. The password stored by + // `kforge secrets apply` is alphanumeric-only so it is safe to + // embed in a shell command without additional escaping. + // + // Prerequisites: + // - The CNPG superuser Secret must exist in the same namespace + // (or copy it there as part of cluster bootstrap). + // - `kforge secrets apply` must have run before this Job. + jobManifest := fmt.Sprintf(`apiVersion: batch/v1 +kind: Job metadata: - name: %s + name: %s-db-init namespace: %s labels: app: %s managed-by: kforge spec: - name: %s - passwordSecret: - name: %s - login: true - superuser: false - createdb: false -`, roleName, env.Namespace, env.FullName, roleName, secretName) - - // The env vars reference the CNPG-managed secret. - // CNPG populates: username, password keys in the secret. - // We assemble DATABASE_URL from the known CNPG host + db name. - cnpgHost := env.CNPGHost - dbURL := fmt.Sprintf("postgresql://$(%s_USER):$(%s_PASSWORD)@%s/%s", - strings.ToUpper(env.FullName), strings.ToUpper(env.FullName), cnpgHost, dbName) + ttlSecondsAfterFinished: 600 + template: + metadata: + labels: + app: %s + spec: + restartPolicy: OnFailure + containers: + - name: db-init + image: postgres:16-alpine + command: + - /bin/sh + - -c + - | + set -e + echo "Ensuring role $DB_USER exists..." + PGPASSWORD="$ADMIN_PASSWORD" psql -h "$DB_HOST" -U "$ADMIN_USER" postgres \ + -c "SELECT 1 FROM pg_roles WHERE rolname = '$DB_USER'" | grep -q 1 || \ + PGPASSWORD="$ADMIN_PASSWORD" psql -h "$DB_HOST" -U "$ADMIN_USER" postgres \ + -c "CREATE ROLE $DB_USER WITH LOGIN PASSWORD '$DB_PASSWORD';" + echo "Syncing password for $DB_USER..." + PGPASSWORD="$ADMIN_PASSWORD" psql -h "$DB_HOST" -U "$ADMIN_USER" postgres \ + -c "ALTER ROLE $DB_USER WITH PASSWORD '$DB_PASSWORD';" + echo "Done." + env: + - name: DB_HOST + value: %s + - name: DB_USER + valueFrom: + secretKeyRef: + name: %s + key: username + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: %s + key: password + - name: ADMIN_USER + valueFrom: + secretKeyRef: + name: %s + key: username + - name: ADMIN_PASSWORD + valueFrom: + secretKeyRef: + name: %s + key: password +`, + env.FullName, env.Namespace, env.FullName, env.FullName, + cnpgHost, + secretName, secretName, + env.CNPGSuperuserSecret, env.CNPGSuperuserSecret, + ) + // DATABASE_URL uses Kubernetes $(VAR_NAME) substitution — DB_USER and + // DB_PASSWORD must be defined earlier in the env list. envVars := []config.EnvVarConfig{ {Name: "DB_HOST", Type: config.EnvVarTypePlain, Value: cnpgHost}, {Name: "DB_PORT", Type: config.EnvVarTypePlain, Value: "5432"}, {Name: "DB_NAME", Type: config.EnvVarTypePlain, Value: dbName}, {Name: "DB_USER", Type: config.EnvVarTypeSecretRef, SecretName: secretName, SecretKey: "username"}, {Name: "DB_PASSWORD", Type: config.EnvVarTypeSecretRef, SecretName: secretName, SecretKey: "password"}, - {Name: "DATABASE_URL", Type: config.EnvVarTypePlain, Value: dbURL}, + {Name: "DATABASE_URL", Type: config.EnvVarTypePlain, + Value: fmt.Sprintf("postgresql://$(DB_USER):$(DB_PASSWORD)@%s/%s", cnpgHost, dbName)}, } return []InfraManifest{ {Name: "cnpg-database", Content: dbManifest, EnvVars: envVars}, - {Name: "cnpg-role", Content: roleManifest}, + {Name: "cnpg-db-init", Content: jobManifest}, }, nil } diff --git a/internal/generator/manifests.go b/internal/generator/manifests.go index 0967fb3..90cd56f 100644 --- a/internal/generator/manifests.go +++ b/internal/generator/manifests.go @@ -181,6 +181,22 @@ func Ingress(env *config.ResolvedEnvironment, _ interpolate.Tokens) (string, err b.WriteString(" nginx.ingress.kubernetes.io/auth-realm: \"Authentication Required\"\n") } + // external-dns annotations: collect all hostnames that need DNS records. + if !env.SkipDNS { + var dnsHosts []string + for _, h := range env.Ingress.Hosts { + if h.DNSRecord { + dnsHosts = append(dnsHosts, h.Hostname) + } + } + if len(dnsHosts) > 0 { + fmt.Fprintf(&b, " external-dns.alpha.kubernetes.io/hostname: \"%s\"\n", strings.Join(dnsHosts, ",")) + if env.DNSTarget != "" { + fmt.Fprintf(&b, " external-dns.alpha.kubernetes.io/target: \"%s\"\n", env.DNSTarget) + } + } + } + b.WriteString("spec:\n") fmt.Fprintf(&b, " ingressClassName: %s\n", env.IngressClass) @@ -405,6 +421,26 @@ func renderStrSlice(ss []string) string { return "[" + strings.Join(quoted, ", ") + "]" } +// Namespace generates a Namespace manifest. Used for preview +// environments where the namespace must be created by kubectl apply. +func Namespace(namespace string, labels map[string]string) string { + var b strings.Builder + b.WriteString("apiVersion: v1\nkind: Namespace\nmetadata:\n") + fmt.Fprintf(&b, " name: %s\n", namespace) + if len(labels) > 0 { + b.WriteString(" labels:\n") + // Stable key order. + keys := make([]string, 0, len(labels)) + for k := range labels { + keys = append(keys, k) + } + for _, k := range keys { + fmt.Fprintf(&b, " %s: %q\n", k, labels[k]) + } + } + return b.String() +} + // render executes a template with data and returns the result. func render(t *template.Template, data any) (string, error) { var buf bytes.Buffer diff --git a/kforge.yml b/kforge.yml index dcfaa60..ee94e2f 100644 --- a/kforge.yml +++ b/kforge.yml @@ -7,14 +7,7 @@ registry: pull_secret: regcred dns: - provider: cloudflare - cloudflare: - api_token: ${CLOUDFLARE_API_TOKEN} - zones: - - name: natelubitz.com - zone_id: ${CF_ZONE_ID_NATELUBITZ} - proxied: false - node_ip: ${KFORGE_NODE_IP} + target: ${KFORGE_NODE_IP} cluster: tls_issuer: letsencrypt-prod diff --git a/pkg/interpolate/interpolate.go b/pkg/interpolate/interpolate.go index b009fe7..0c3e188 100644 --- a/pkg/interpolate/interpolate.go +++ b/pkg/interpolate/interpolate.go @@ -119,3 +119,20 @@ func Slug(s string) string { func HostSlug(hostname string) string { return Slug(strings.ReplaceAll(hostname, ".", "-")) } + +// PGIdentifier converts a kforge name into a valid unquoted +// PostgreSQL identifier: lowercase, hyphens and spaces become +// underscores, all other non-alphanumeric characters are dropped. +// "prod-my-tenant-myapp" → "prod_my_tenant_myapp" +func PGIdentifier(s string) string { + var b strings.Builder + for _, r := range strings.ToLower(s) { + switch { + case r >= 'a' && r <= 'z', r >= '0' && r <= '9', r == '_': + b.WriteRune(r) + case r == '-', r == ' ': + b.WriteRune('_') + } + } + return strings.Trim(b.String(), "_") +} diff --git a/readme.md b/readme.md index a8ac16d..c15cb45 100644 --- a/readme.md +++ b/readme.md @@ -1,8 +1,8 @@ # kforge -**kforge** eliminates Kubernetes boilerplate. You define your app once in a `kforge.yml` file — environments, infrastructure, ingress, TLS, DNS — and kforge generates production-ready flat manifests on every CI run. Nothing is committed to your repo except the config. +**kforge** eliminates Kubernetes boilerplate. You define your app once in a `kforge.yml` file — environments, infrastructure, ingress, TLS, DNS — and kforge generates production-ready flat manifests on every CI run. Nothing is committed to your repo except the config and generated workflows. -Built for self-hosted MicroK8s, Gitea Actions, Cloudflare DNS, cert-manager, and CNPG — but designed to be extended. +Built for self-hosted MicroK8s, Gitea Actions, external-dns, cert-manager, and CNPG — but designed to be extended. --- @@ -10,12 +10,14 @@ Built for self-hosted MicroK8s, Gitea Actions, Cloudflare DNS, cert-manager, and 1. Add a `kforge.yml` to your repo root describing your app, environments, and infrastructure. 2. kforge generates Kubernetes YAML at CI time — Service, Deployment, Ingress, cert-manager Certificates, CronJobs, and infrastructure (database, cache, storage, queue, search). -3. Generated manifests are applied to the cluster and discarded. Only `kforge.yml` is committed. +3. Generated manifests are applied to the cluster and discarded. Only `kforge.yml` and generated workflow files are committed. ``` kforge.yml → kforge generate → kubectl apply → cluster ``` +DNS records are managed by **external-dns** running in the cluster — kforge writes the appropriate annotations on the Ingress and external-dns creates the records automatically. TLS is handled by **cert-manager** reading those same Certificate CRs. + --- ## Installation @@ -57,13 +59,7 @@ registry: url: registry.yourdomain.com dns: - provider: cloudflare - cloudflare: - api_token: ${CLOUDFLARE_API_TOKEN} - zones: - - name: yourdomain.com - zone_id: ${CF_ZONE_ID_YOURDOMAIN} - node_ip: ${KFORGE_NODE_IP} + target: ${KFORGE_NODE_IP} # your cluster node's public IP cluster: tls_issuer: letsencrypt-prod @@ -94,7 +90,7 @@ environments: hosts: - hostname: app-staging.yourdomain.com tls: true - dns_record: true + dns_record: true # external-dns creates this record auth: enabled: true users: @@ -131,19 +127,101 @@ kforge secrets list kforge generate --env production --dry-run ``` -**4. Generate your Gitea Actions workflow:** +**4. Generate your Gitea Actions workflows:** ```bash kforge gitea-actions > .gitea/workflows/deploy.yml +kforge gitea-preview > .gitea/workflows/preview.yml # optional — PR preview environments ``` --- +## Adding kforge to an existing project + +### Step 1 — Add `kforge.yml` and generate workflows + +Add `kforge.yml` to your repo root (see Quick start above), then: + +```bash +kforge gitea-actions # → .gitea/workflows/deploy.yml +kforge gitea-preview # → .gitea/workflows/preview.yml (if preview.enabled: true) +``` + +Commit `kforge.yml` and both generated workflow files. + +### Step 2 — Gitea secrets + +**Org-level** (set once, shared across all repos): + +| Secret | Value | +|---|---| +| `DOCKER_USERNAME` | Registry login username | +| `DOCKER_PASSWORD` | Registry login password/token | +| `KFORGE_NODE_IP` | Your cluster's public node IP (used as the external-dns record target) | + +**Repo-level** (per project): + +| Secret | How to get it | +|---|---| +| `KUBE_HOST` | `kubectl config view --raw -o jsonpath='{.clusters[0].cluster.server}'` | +| `KUBE_CERTIFICATE` | `kubectl config view --raw -o jsonpath='{.clusters[0].cluster.certificate-authority-data}'` | +| `KUBE_TOKEN` | See below | + +Create a long-lived deploy service account: + +```bash +kubectl create serviceaccount kforge-deployer -n kube-system +kubectl create clusterrolebinding kforge-deployer \ + --clusterrole=cluster-admin \ + --serviceaccount=kube-system:kforge-deployer +kubectl create token kforge-deployer -n kube-system --duration=8760h +``` + +### Step 3 — Cluster prerequisites + +These must already be running on your cluster: + +| Component | Purpose | +|---|---| +| **NGINX ingress controller** | Handles `ingress_class: nginx` | +| **cert-manager** + `ClusterIssuer` named `letsencrypt-prod` | Issues TLS certificates | +| **external-dns** | Reads Ingress annotations, creates DNS records | +| **CNPG operator** | Required if `infrastructure.database` is enabled | +| **`regcred` imagePullSecret** | Must exist in each environment namespace | + +For external-dns, configure it with your DNS provider and `--source=ingress`. kforge writes the target annotation automatically from `dns.target`. + +Create `regcred` in each namespace: + +```bash +kubectl create secret docker-registry regcred \ + --docker-server=registry.yourdomain.com \ + --docker-username= \ + --docker-password= \ + -n production +``` + +### Step 4 — First deploy + +The workflow is fully automated after setup, but the very first time: + +1. Create the namespace: `kubectl create namespace production` +2. Create `regcred` in that namespace (above). +3. If using CNPG database, make the CNPG superuser Secret available in the namespace. The default secret name is `cnpg-main-superuser` — copy it from wherever your CNPG cluster lives: + ```bash + kubectl get secret cnpg-main-superuser -n cnpg-system -o yaml \ + | sed 's/namespace: cnpg-system/namespace: production/' \ + | kubectl apply -f - + ``` +4. Push to `main`. The workflow runs `kforge secrets apply` (creates credentials), `kforge generate` (writes manifests), and `kubectl apply`. + +--- + ## CLI reference ### `kforge validate` -Parses `kforge.yml`, checks structural correctness, and verifies all required secrets are present in the current environment. Exits non-zero if anything is wrong — use this as the first step in CI to fail fast before touching the cluster. +Parses `kforge.yml`, checks structural correctness, and verifies all required secrets are present in the current environment. Use this as the first step in CI to fail fast before touching the cluster. ```bash kforge validate @@ -161,21 +239,24 @@ kforge generate # all environments kforge generate --env staging # one environment kforge generate --env production --dry-run kforge generate --env production --output .kube/ +kforge generate --pr-number 42 # preview environment for PR #42 ``` **Output files per environment:** -| File | Contents | -| --------------------------------- | ---------------------------------------------------- | -| `{env}-core.yaml` | Service, Deployment, Ingress, Certificates, CronJobs | -| `{env}-infra-cnpg-database.yaml` | CNPG Database CR | -| `{env}-infra-cnpg-role.yaml` | CNPG DatabaseRole CR | -| `{env}-infra-cache.yaml` | Valkey/Redis Deployment or StatefulSet | -| `{env}-infra-cache-svc.yaml` | Cache Service | -| `{env}-infra-storage.yaml` | Minio Deployment | -| `{env}-infra-queue-nats.yaml` | NATS Deployment | -| `{env}-infra-search.yaml` | Meilisearch Deployment | -| `{env}-infra-servicemonitor.yaml` | Prometheus ServiceMonitor CR | +| File | Contents | +|---|---| +| `{env}-core.yaml` | Service, Deployment, Ingress, Certificates, CronJobs | +| `{env}-infra-cnpg-database.yaml` | CNPG Database CR | +| `{env}-infra-cnpg-db-init.yaml` | Job that creates the PostgreSQL role and syncs password | +| `{env}-infra-cache.yaml` | Valkey/Redis Deployment or StatefulSet | +| `{env}-infra-cache-svc.yaml` | Cache Service | +| `{env}-infra-storage.yaml` | Minio Deployment | +| `{env}-infra-queue-nats.yaml` | NATS Deployment | +| `{env}-infra-search.yaml` | Meilisearch Deployment | +| `{env}-infra-servicemonitor.yaml` | Prometheus ServiceMonitor CR | + +When `--pr-number` is given, the output also includes a `Namespace` manifest so `kubectl apply` is self-contained — no separate namespace creation step needed. Infrastructure env vars (`DATABASE_URL`, `CACHE_URL`, `STORAGE_ENDPOINT`, etc.) are automatically injected into the Deployment — you don't wire these up manually. @@ -193,20 +274,17 @@ Example output: ``` ── Gitea org secret ── - DOCKER_USERNAME ✗ missing - CLOUDFLARE_API_TOKEN ✓ set - CF_ZONE_ID_YOURDOMAIN_COM ✓ set - KFORGE_NODE_IP ✓ set - SOPS_AGE_KEY ✗ missing + DOCKER_USERNAME ✗ missing + KFORGE_NODE_IP ✓ set ── Gitea repo secret ── - KUBE_HOST ✓ set - KUBE_TOKEN ✓ set - KUBE_CERTIFICATE ✓ set + KUBE_HOST ✓ set + KUBE_TOKEN ✓ set + KUBE_CERTIFICATE ✓ set ── Cluster secret (auto-generated) ── - prod-my-org-my-app-db-credentials — managed by kforge - prod-my-org-my-app-cache-credentials — managed by kforge + prod-my-org-my-app-db-credentials — managed by kforge + prod-my-org-my-app-cache-credentials — managed by kforge ``` --- @@ -218,27 +296,26 @@ Generates secure random credentials and creates Kubernetes Secrets in the cluste ```bash kforge secrets apply --env staging kforge secrets apply --env production --force # rotates all credentials +kforge secrets apply --pr-number 42 # apply secrets for PR preview #42 ``` -For basic auth secrets, kforge prints the generated passwords once at apply time. Save them — they are not stored anywhere else. +**What gets created:** + +| Secret | Contents | +|---|---| +| `{full_name}-db-credentials` | PostgreSQL `username` (derived from app name) + random alphanumeric `password` | +| `{full_name}-basic-auth` | htpasswd entries for ingress basic auth | +| `{full_name}-cache-credentials` | Valkey/Redis password | +| `{full_name}-storage-credentials` | Minio access key + secret key | +| `{full_name}-queue-credentials` | RabbitMQ username + password (NATS needs no credentials) | +| `{full_name}-search-credentials` | Meilisearch master key | + +For basic auth secrets, kforge prints the generated passwords once at apply time — save them. Requires `KUBE_HOST`, `KUBE_TOKEN`, and `KUBE_CERTIFICATE` to be set (Gitea injects these automatically during CI). --- -### `kforge dns ensure` - -Creates or updates Cloudflare DNS A records for all ingress hosts with `dns_record: true`. Idempotent — no-ops if the record already points to the correct IP. - -```bash -kforge dns ensure --env staging -kforge dns ensure --env staging --env production -``` - -Requires `CLOUDFLARE_API_TOKEN` and `KFORGE_NODE_IP` to be set. - ---- - ### `kforge gitea-actions` Generates a complete `.gitea/workflows/deploy.yml` for this app. Re-run whenever you add environments or change deploy configuration. @@ -251,14 +328,32 @@ kforge gitea-actions --branch main --env staging --env production The generated workflow runs these steps for each environment, in order: -1. Build and push Docker image +1. Build and push Docker image (tagged with git SHA) 2. `kforge validate` -3. `kforge secrets apply` — creates missing cluster secrets -4. `kforge dns ensure` — creates missing DNS records -5. `kforge generate` — writes manifests to `.kforge-out/` -6. `kubectl apply` — applies core manifests -7. `kubectl apply` — applies infra manifests -8. `kubectl rollout restart` — triggers rolling update +3. `kforge secrets apply` — creates any missing cluster secrets +4. `kforge generate` — writes manifests to `.kforge-out/` +5. `kubectl apply` — applies core manifests (Service, Deployment, Ingress, Certs) +6. `kubectl apply` — applies infrastructure manifests (database, cache, etc.) +7. `kubectl rollout restart` — triggers rolling update + +DNS records are created automatically by external-dns when the Ingress is applied — no separate DNS step needed. + +--- + +### `kforge gitea-preview` + +Generates `.gitea/workflows/preview.yml` that deploys an ephemeral environment for each pull request. + +```bash +kforge gitea-preview +kforge gitea-preview --output .gitea/workflows/preview.yml +``` + +Requires `preview.enabled: true` in `kforge.yml`. See the [preview environments](#preview-environments) section below. + +The generated workflow: +- **On PR open / sync**: Builds a `:pr-{N}` tagged image, applies secrets to a new namespace (`preview-pr-{N}`), generates manifests (including a `Namespace` resource), and deploys. +- **On PR close**: Deletes the `preview-pr-{N}` namespace, removing all preview resources automatically. --- @@ -268,16 +363,16 @@ The generated workflow runs these steps for each environment, in order: kforge generates resource names using the pattern `{env_prefix}-{tenant}-{name}` (e.g. `prod-my-org-my-app`). Use `${tokens}` anywhere in string values to reference resolved fields: -| Token | Resolves to | -| --------------- | ------------------------------------------- | -| `${name}` | `meta.name` | -| `${tenant}` | `meta.tenant` | -| `${env}` | current environment key | +| Token | Resolves to | +|---|---| +| `${name}` | `meta.name` | +| `${tenant}` | `meta.tenant` | +| `${env}` | current environment key | | `${env_prefix}` | short env prefix (first 4 chars, or custom) | -| `${full_name}` | `{env_prefix}-{tenant}-{name}` | -| `${namespace}` | resolved namespace for the environment | +| `${full_name}` | `{env_prefix}-{tenant}-{name}` | +| `${namespace}` | resolved namespace for the environment | -Environment variables (e.g. `${CLOUDFLARE_API_TOKEN}`) are resolved from the CI process environment at generation time — never hardcode secrets in `kforge.yml`. +Environment variables (e.g. `${KFORGE_NODE_IP}`) are resolved from the CI process environment at generation time — never hardcode secrets in `kforge.yml`. --- @@ -285,10 +380,10 @@ Environment variables (e.g. `${CLOUDFLARE_API_TOKEN}`) are resolved from the CI ```yaml meta: - name: my-app # required — short app name, lowercase, hyphens ok - tenant: my-org # required — org/tenant identifier - name_override: ~ # optional — override the full generated resource name - previous_name: ~ # optional — set when renaming; kforge patches rather than recreates + name: my-app # required — short app name, lowercase, hyphens ok + tenant: my-org # required — org/tenant identifier + name_override: ~ # optional — override the full generated resource name + previous_name: ~ # optional — set when renaming; kforge patches rather than recreates ``` --- @@ -297,9 +392,9 @@ meta: ```yaml registry: - url: registry.yourdomain.com # default: registry.natelubitz.com - repository: my-org/my-app # default: {tenant}/{name} - pull_secret: regcred # default: regcred + url: registry.yourdomain.com # default: registry.natelubitz.com + repository: my-org/my-app # default: {tenant}/{name} + pull_secret: regcred # default: regcred ``` Override per-environment by adding a `registry:` block under the environment. @@ -308,20 +403,20 @@ Override per-environment by adding a `registry:` block under the environment. ### `dns` -```yaml -dns: - provider: cloudflare # currently supported: cloudflare - cloudflare: - api_token: ${CLOUDFLARE_API_TOKEN} - zones: - - name: yourdomain.com - zone_id: ${CF_ZONE_ID_YOURDOMAIN} - proxied: false # false = DNS-only, required for cert-manager DNS-01 - node_ip: ${KFORGE_NODE_IP} # IP for new A records - skip_dns: false # true to disable all DNS management +DNS records are managed by **external-dns** inside the cluster. kforge writes annotations on the Ingress resource for each host with `dns_record: true`: + +``` +external-dns.alpha.kubernetes.io/hostname: "app.yourdomain.com" +external-dns.alpha.kubernetes.io/target: "" ``` -kforge matches each ingress hostname to the correct zone by longest-suffix match — add one zone entry per domain you own. +```yaml +dns: + target: ${KFORGE_NODE_IP} # value for the external-dns target annotation + skip_dns: false # true = don't write external-dns annotations +``` + +`target` supports token interpolation — `${KFORGE_NODE_IP}` is the most common value, resolved from the `KFORGE_NODE_IP` Gitea org secret at generate time. --- @@ -329,11 +424,13 @@ kforge matches each ingress hostname to the correct zone by longest-suffix match ```yaml cluster: - tls_issuer: letsencrypt-prod # default: letsencrypt-prod - ingress_class: nginx # default: nginx + tls_issuer: letsencrypt-prod # default: letsencrypt-prod + ingress_class: nginx # default: nginx cnpg: - host: cnpg-main-rw.default.svc.cluster.local # your CNPG cluster service - namespace_pattern: "${env}" # default: environment key + host: cnpg-main-rw.default.svc.cluster.local # your CNPG cluster's read-write service + cluster_name: cnpg-main # default: cnpg-main + superuser_secret: cnpg-main-superuser # default: cnpg-main-superuser + namespace_pattern: "${env}" # default: environment key ``` --- @@ -377,17 +474,17 @@ Define infrastructure at the root level and it applies to **all environments** b ```yaml infrastructure: database: - provider: cnpg # only supported provider currently + provider: cnpg # only supported provider currently cache: provider: valkey # valkey (recommended) | redis mode: standalone # standalone | cluster replicas: 1 storage: enabled: false - provider: minio # standalone | distributed + provider: minio # standalone | distributed queue: enabled: false - provider: nats # nats (recommended, ~20MB) | rabbitmq (~200MB) + provider: nats # nats (recommended, ~20MB) | rabbitmq (~200MB) search: enabled: false provider: meilisearch @@ -415,19 +512,53 @@ environments: infrastructure: cache: mode: cluster - replicas: 3 # provider: valkey inherited from root + replicas: 3 # provider: valkey inherited from root ``` **Injected env vars per service** (automatically added to your Deployment): -| Service | Env vars injected | -| ---------------- | ------------------------------------------------------------------------- | -| database (CNPG) | `DATABASE_URL`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` | -| cache | `CACHE_URL`, `CACHE_HOST`, `CACHE_PORT`, `CACHE_PASSWORD` | -| storage | `STORAGE_ENDPOINT`, `STORAGE_ACCESS_KEY`, `STORAGE_SECRET_KEY` | -| queue (NATS) | `QUEUE_URL`, `QUEUE_HOST` | -| queue (RabbitMQ) | `QUEUE_URL`, `QUEUE_USER`, `QUEUE_PASSWORD` | -| search | `SEARCH_URL`, `SEARCH_MASTER_KEY` | +| Service | Env vars injected | +|---|---| +| database (CNPG) | `DATABASE_URL`, `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` | +| cache | `CACHE_URL`, `CACHE_HOST`, `CACHE_PORT`, `CACHE_PASSWORD` | +| storage | `STORAGE_ENDPOINT`, `STORAGE_ACCESS_KEY`, `STORAGE_SECRET_KEY` | +| queue (NATS) | `QUEUE_URL`, `QUEUE_HOST` | +| queue (RabbitMQ) | `QUEUE_URL`, `QUEUE_USER`, `QUEUE_PASSWORD` | +| search | `SEARCH_URL`, `SEARCH_MASTER_KEY` | + +#### CNPG database details + +For a centralized CNPG cluster, kforge generates two resources: + +1. A **`Database` CR** — declaratively manages the database lifecycle (CNPG v1.22+). +2. A **`db-init` Job** — runs on every deploy to create the PostgreSQL role (if it doesn't exist) and sync its password from the `{full_name}-db-credentials` Secret. + +`kforge secrets apply` must run before the Job so the Secret exists. The PostgreSQL username is derived from the app's full name (`prod_my_org_my_app`). The password is alphanumeric only, which keeps the init Job shell script simple and safe. + +The `cnpg-main-superuser` Secret (created by CNPG for the cluster) must be present in the target namespace. Copy it once during cluster bootstrap or namespace creation. + +--- + +### `preview` + +Enables PR preview environments. Run `kforge gitea-preview` to generate the workflow. + +```yaml +preview: + enabled: true + base_environment: staging # inherit infra and env vars from this env + namespace_prefix: preview-pr # namespace = preview-pr-{PR_NUMBER} + hostname_template: "pr-${PR_NUMBER}.${name}.yourdomain.com" +``` + +Supported tokens in `hostname_template`: `${PR_NUMBER}`, `${name}`, `${tenant}`. + +On PR open/sync, kforge deploys the app to a `preview-pr-{N}` namespace with: +- Image tagged `:pr-{N}` (built from the PR branch) +- Hostname from the template +- Infrastructure and env vars inherited from `base_environment` + +On PR close, the entire namespace is deleted. --- @@ -438,8 +569,8 @@ environments: production: namespace: production replicas: 1 - image_tag: latest # override with --set image_tag=$SHA in CI - env_prefix: prod # default: first 4 chars of env key + image_tag: latest # override with --set image_tag=$SHA in CI + env_prefix: prod # default: first 4 chars of env key env_vars: - name: API_URL @@ -447,24 +578,24 @@ environments: value: https://api.yourdomain.com - name: SOME_SECRET - type: secret_ref # pull from an existing Kubernetes Secret + type: secret_ref # pull from an existing Kubernetes Secret secret_name: my-secrets secret_key: some_secret - name: FEATURE_FLAG - type: configmap_ref # pull from a ConfigMap + type: configmap_ref # pull from a ConfigMap configmap_name: my-config configmap_key: feature_flag ingress: hosts: - hostname: app.yourdomain.com - tls: true # kforge generates a cert-manager Certificate - dns_record: true # kforge creates a Cloudflare A record + tls: true # kforge generates a cert-manager Certificate + dns_record: true # external-dns creates this record auth: - enabled: false # enable for staging/dev to protect unreleased work + enabled: false # enable for staging/dev to protect unreleased work users: - - yourname # passwords are auto-generated by kforge secrets apply + - yourname # passwords are auto-generated by kforge secrets apply infrastructure: # shallow merge on top of root — only override what differs @@ -476,7 +607,7 @@ environments: - name: cleanup schedule: "0 2 * * *" command: ["node", "scripts/cleanup.js"] - inherit_env: true # inherits all deployment env vars + inherit_env: true # inherits all deployment env vars env_vars: - name: BATCH_SIZE value: "500" @@ -492,8 +623,8 @@ environments: concurrency_policy: Forbid lifecycle: - delete: false # if true + previous_name set, deletes old resources - delete_grace_seconds: 300 # 5-minute countdown before deletion runs in CI + delete: false # if true + previous_name set, deletes old resources + delete_grace_seconds: 300 ``` --- @@ -506,37 +637,34 @@ kforge works with three categories of secrets, each living in the right place fo Set once at the organisation level; available to every repo automatically. -| Secret | Purpose | -| ---------------------- | ---------------------------------------------- | -| `DOCKER_USERNAME` | Registry authentication | -| `DOCKER_PASSWORD` | Registry authentication | -| `CLOUDFLARE_API_TOKEN` | DNS record management (Zone:Read + DNS:Edit) | -| `CF_ZONE_ID_{DOMAIN}` | One per zone, e.g. `CF_ZONE_ID_YOURDOMAIN_COM` | -| `KFORGE_NODE_IP` | MicroK8s node IP for DNS A records | -| `SOPS_AGE_KEY` | Decrypts `.kforge/secrets.enc.yml` | +| Secret | Purpose | +|---|---| +| `DOCKER_USERNAME` | Registry authentication | +| `DOCKER_PASSWORD` | Registry authentication | +| `KFORGE_NODE_IP` | Cluster node IP — written as the external-dns target annotation | ### Category B — Gitea repo secrets Per-repo, since different apps may deploy to different clusters. -| Secret | Purpose | -| ------------------ | ----------------------------- | -| `KUBE_HOST` | Kubernetes API server URL | -| `KUBE_TOKEN` | Service account token | +| Secret | Purpose | +|---|---| +| `KUBE_HOST` | Kubernetes API server URL | +| `KUBE_TOKEN` | Service account token | | `KUBE_CERTIFICATE` | Base64-encoded CA certificate | ### Category C — Cluster secrets (auto-generated) Created by `kforge secrets apply`. Never appear in Gitea or in `kforge.yml`. -| Secret name | Contents | -| --------------------------------- | --------------------------------- | -| `{full_name}-db-credentials` | CNPG-managed database credentials | -| `{full_name}-basic-auth` | htpasswd for ingress basic auth | -| `{full_name}-cache-credentials` | Valkey/Redis password | -| `{full_name}-storage-credentials` | Minio access/secret keys | -| `{full_name}-queue-credentials` | RabbitMQ credentials | -| `{full_name}-search-credentials` | Meilisearch master key | +| Secret name | Contents | +|---|---| +| `{full_name}-db-credentials` | PostgreSQL `username` + alphanumeric `password` | +| `{full_name}-basic-auth` | htpasswd for ingress basic auth | +| `{full_name}-cache-credentials` | Valkey/Redis password | +| `{full_name}-storage-credentials` | Minio access/secret keys | +| `{full_name}-queue-credentials` | RabbitMQ credentials | +| `{full_name}-search-credentials` | Meilisearch master key | Run `kforge secrets list` at any time to see the full checklist with live status for your current repo. @@ -549,9 +677,7 @@ kforge.yml ← your app config (committed) .gitea/ workflows/ deploy.yml ← generated by kforge gitea-actions (committed) -.kforge/ - secrets.enc.yml ← SOPS-encrypted sensitive config (committed) - kforge.age ← age private key (NEVER committed — goes in Gitea as SOPS_AGE_KEY) + preview.yml ← generated by kforge gitea-preview (committed, optional) ``` Generated manifests (`.kforge-out/`) are never committed — they are created at CI time and discarded after `kubectl apply`.