Skip to main content

Best practices

Table of Contents​

  1. Repository structure
  2. Secrets management
  3. Promotion between environments
  4. Monitoring and debugging
  5. Production checklist
  6. Hands-on exercises

1 - Repository structure​

gitops-fleet/
├── clusters/
│   ├── dev/
│   │   ├── flux-system/
│   │   └── apps.yaml
│   ├── staging/
│   │   ├── flux-system/
│   │   └── apps.yaml
│   └── production/
│       ├── flux-system/
│       └── apps.yaml
├── infrastructure/
│   ├── sources/           # HelmRepository, GitRepository
│   ├── controllers/       # cert-manager, ingress, etc.
│   └── configs/           # ConfigMaps partagés
└── apps/
    ├── base/
    │   ├── app-1/
    │   └── app-2/
    └── overlays/
        ├── dev/
        ├── staging/
        └── production/

Naming convention​

# Préfixer par l'environnement
metadata:
  name: production-nginx
  # ou
  name: nginx
  namespace: production

Standard labels​

metadata:
  labels:
    app.kubernetes.io/name: nginx
    app.kubernetes.io/component: frontend
    environment: production
    team: platform

🔝 Back to table of contents​

2 - Secrets management​

# Installer SOPS
brew install sops

# Créer une clé age
age-keygen -o age.key

# Chiffrer
sops --encrypt --age age1xxx... secret.yaml > secret.enc.yaml
# Kustomization avec SOPS
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
spec:
  decryption:
    provider: sops
    secretRef:
      name: sops-age

Sealed Secrets​

# Installer kubeseal
brew install kubeseal

# Chiffrer
kubeseal --format yaml < secret.yaml > sealed-secret.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: my-secret
spec:
  encryptedData:
    password: AgBy8hCi...

External Secrets​

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-secret
spec:
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: my-secret
  data:
    - secretKey: password
      remoteRef:
        key: secret/myapp
        property: password

Comparison​

SolutionComplexitySecurityRotation
SOPSMediumExcellentManual
Sealed SecretsLowGoodManual
External SecretsHighExcellentAutomatic

🔝 Back to table of contents​

3 - Promotion between environments​

Git branches pattern​

Kustomize overlays pattern​

# apps/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 1

# apps/overlays/production/kustomization.yaml
patches:
  - patch: |
      - op: replace
        path: /spec/replicas
        value: 5

ImagePolicy per environment pattern​

# Dev : dernier tag
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
  name: my-app-dev
spec:
  policy:
    alphabetical:
      order: desc
---
# Prod : semver stable
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
  name: my-app-prod
spec:
  policy:
    semver:
      range: ">=1.0.0"

🔝 Back to table of contents​

4 - Monitoring and debugging​

Debug commands​

# État général
flux check

# Toutes les ressources
flux get all -A

# Logs d'un controller
flux logs --kind=Kustomization --name=my-app

# Events récents
kubectl get events -n flux-system --sort-by=.lastTimestamp

# Reconcilier manuellement
flux reconcile kustomization my-app --with-source

Prometheus metrics​

# ServiceMonitor pour Flux
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: flux
  namespace: flux-system
spec:
  selector:
    matchLabels:
      app.kubernetes.io/part-of: flux
  endpoints:
    - port: http-prom

Important alerts​

# Alertmanager rule
- alert: FluxReconciliationFailure
  expr: gotk_reconcile_condition{status="False",type="Ready"} == 1
  for: 10m
  labels:
    severity: critical
  annotations:
    summary: "Flux reconciliation failing"

Grafana dashboard​

Flux provides an official Grafana dashboard:

  • ID: 16714 (Flux Cluster Stats)
  • ID: 16716 (Flux Control Plane)

🔝 Back to table of contents​

5 - Production checklist​

Before going to production​

  • Bootstrap with private repo and SSH
  • SOPS or another solution for secrets
  • Notifications Slack/Teams configured
  • Monitoring Prometheus + Grafana
  • RBAC multi-tenancy if needed
  • NetworkPolicies for flux-system
  • ResourceQuotas for tenants
# Kustomization production
spec:
  interval: 5m
  retryInterval: 2m
  timeout: 5m
  prune: true
  wait: true
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: my-app

Security​

# Restreindre les sources autorisées
# (au niveau du namespace tenant)
spec:
  serviceAccountName: tenant-flux
  targetNamespace: tenant-namespace

Backup​

# Exporter toutes les ressources Flux
flux export source git --all > sources.yaml
flux export kustomization --all > kustomizations.yaml
flux export helmrelease --all > helmreleases.yaml

🔝 Back to table of contents​

6 - Hands-on exercises​

Exercise: Complete GitOps structure​

# Créer cette structure
my-gitops/
├── clusters/
│   └── demo/
│       ├── flux-system/
│       │   ├── gotk-components.yaml
│       │   ├── gotk-sync.yaml
│       │   └── kustomization.yaml
│       └── apps.yaml
├── infrastructure/
│   └── sources/
│       └── bitnami.yaml
└── apps/
    └── nginx/
        ├── namespace.yaml
        ├── release.yaml
        └── kustomization.yaml

Quiz​

Q1. Which secrets solution is recommended for Flux?

Answer

SOPS with age or GPG is the recommended solution because:

  • Natively integrated in Flux
  • Encrypted secrets are versioned in Git
  • No external dependency at runtime

Alternative: External Secrets for dynamic secrets from a vault.

Q2. How do you force an immediate reconciliation?

Answer
# Réconcilier une Kustomization avec sa source
flux reconcile kustomization my-app --with-source

# Ou juste la Kustomization
flux reconcile kustomization my-app

# Ou via annotation
kubectl annotate kustomization my-app \
  reconcile.fluxcd.io/requestedAt="$(date +%s)" \
  -n flux-system

🔝 Back to table of contents​

Key takeaways​

  • Clear structure: clusters/, infrastructure/, apps/
  • SOPS for secrets in Git
  • Kustomize overlays for promotion
  • Monitoring: Prometheus + Flux dashboards
  • Production checklist before deployment

← Previous chapter | Next chapter: Exercises →