kubernetes-topics-custom-controllers

• Doc-Ref

Contents

I’ll help you understand the different approaches to handling variables in Kubernetes manifests using Kustomize.

Here’s a comparative analysis of the main methods:

Characteristics and Trade-offss

ConfigMapGenerator:

• Best for application configuration
• Automatically handles updates and rolling deployments
• Limited to ConfigMap format
• Cannot handle sensitive data

SecretGenerator:

• Similar to ConfigMapGenerator but for sensitive data
• Built-in base64 encodings
• Supports multiple formats (files, literals, env files)
• Requires careful handling of secret values

Vars:

• Useful for cross-resource references
• Limited to specific fields (names, labels)
• Not as flexible as replacements
• Good for maintaining consistency

Replacements:

• Most flexible option
• Can modify any field in any resource
• More complex configuration
• Potential for errors if not carefully managed

Let’s look at examples for each method:

Best Practices and Tips

• Use ConfigMapGenerator and SecretGenerator for most application configurations:

  • They provide automatic hash suffixes for updates
  • Handle format conversion automatically
  • Integrate well with Kubernetes native resources

• Use Replacements for complex scenarios:

  • When you need to modify specific fields in resources
  • For cross-resource value propagation
  • When dealing with dynamic values

• Use Vars sparingly:

  • Best for simple name references
  • When you need to ensure consistency across resources
  • For service discovery scenarios

• Consider environment overlays:

  • Create base configurations
  • Use overlays for environment-specific values
  • Maintain clear separation of concerns

• Security considerations:

  • Never commit secrets to version control
  • Use SecretGenerator with external secret sources
  • Consider using sealed secrets or external secret management solutions

# 1. ConfigMapGenerator Example
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

configMapGenerator:
- name: app-config
  literals:
  - DB_HOST=mysql.default.svc.cluster.local
  - DB_PORT=3306
  files:
  - configs/app.properties

---
# Deployment using ConfigMap
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
      - name: app
        envFrom:
        - configMapRef:
            name: app-config

# 2. SecretGenerator Example
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

secretGenerator:
- name: app-secrets
  files:
  - secrets/api.key
  literals:
  - DB_PASSWORD=secretvalue

---
# Deployment using Secret
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
      - name: app
        envFrom:
        - secretRef:
            name: app-secrets

# 3. Vars Example
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

vars:
- name: SERVICE_NAME
  objref:
    kind: Service
    name: my-service
    apiVersion: v1
  fieldref:
    fieldpath: metadata.name

---
# Service referenced by Vars
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  ports:
  - port: 8080

# 4. Replacements Example
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

replacements:
- source:
    kind: ConfigMap
    name: env-config
    fieldPath: data.REPLICA_COUNT
  targets:
  - select:
      kind: Deployment
      name: my-app
    fieldPaths:
    - spec.replicas

---
# ConfigMap with replacement values
apiVersion: v1
kind: ConfigMap
metadata:
  name: env-config
data:
  REPLICA_COUNT: "3"

# 5. Values/ValuesFrom Example
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

configMapGenerator:
- name: app-config
  behavior: merge
  envs:
  - configs/.env

---
# configs/.env
APP_PORT=8080
APP_NAME=myapp

# 6. Patches Example
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

patches:
- patch: |-
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: my-app
    spec:
      template:
        spec:
          containers:
          - name: app
            resources:
              requests:
                memory: "256Mi"
              limits:
                memory: "512Mi"
  target:
    kind: Deployment
    name: my-app

# Strategic merge patch
patchesStrategicMerge:
- |-
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: my-app
  spec:
    replicas: 3

Here are the Kustomize best practices with simple explanations and practical examples:

• Base and Overlays

  • What: Split your config into a base (common stuff) and overlays (environment-specific changes)
  • Example:

  # Project structure
  base/
    kustomization.yaml    # Common config
    deployment.yaml
  overlays/
    dev/
      kustomization.yaml  # Dev-specific changes
    prod/
      kustomization.yaml  # Prod-specific changes

• Unique Names

  • What: Add prefixes/suffixes to avoid name conflicts
  • Example:

  # kustomization.yaml
  namePrefix: dev-
  nameSuffix: -v1
  resources:
  - deployment.yaml

• Common Labels

  • What: Use consistent labels to organize resources
  • Example:

  # kustomization.yaml
  commonLabels:
    app: myapp
    env: prod
    team: frontend

• ConfigMap Generation

  • What: Generate ConfigMaps from files instead of writing them manually
  • Example:

  # kustomization.yaml
  configMapGenerator:
  - name: app-config
    files:
    - config.properties
    - settings.json

• Resource Management

  • What: Keep related resources together in folders
  • Example:

  # Project structure
  base/
    frontend/
      deployment.yaml
      service.yaml
    backend/
      deployment.yaml
      service.yaml

• Resource Limits

  • What: Always set CPU and memory limits
  • Example:

  # deployment.yaml
  resources:
    limits:
      memory: "256Mi"
      cpu: "500m"
    requests:
      memory: "128Mi"
      cpu: "250m"

• Image Management

  • What: Control image versions centrally
  • Example:

  # kustomization.yaml
  images:
  - name: nginx
    newName: my-registry/nginx
    newTag: v1.2.3

• Environment Variables

  • What: Use ConfigMaps for env vars instead of hardcoding
  • Example:

  # deployment.yaml
  envFrom:
  - configMapRef:
      name: app-config

• Patches

• What: Use patches to modify specific parts of resources
• Example:

# kustomization.yaml
patchesStrategicMerge:
- update-memory.yaml  # Only changes memory settings

• Documentation

• What: Add README files to explain your setup
• Example:

# dev/README.md
This overlay contains development-specific settings:
- Lower resource limits
- Debug mode enabled
- Local database connection