Kubernetes Deployment Strategies for SaaS

The default Kubernetes rolling update — replace pods gradually until all are on the new version — is a reasonable starting point, but it makes rollout decisions based on pod readiness, not on whether the new version is actually behaving correctly in production.

Progressive delivery

Progressive deployment routes a small percentage of real traffic to the new version first, watches production signal (error rate, latency, saturation), and only proceeds to full rollout if that signal stays healthy. If it doesn't, the pipeline rolls back automatically — no one has to notice the problem first.

This is a meaningfully different guarantee than "the new pods started successfully." A pod can be ready and still be serving 500s under real traffic patterns that didn't show up in a health check.

What the gate should actually check

The value of progressive delivery is entirely in what you gate on. Useful signals:

• Error rate on the new version compared to the baseline, not just an absolute threshold.
• Latency percentiles (p95/p99), since averages hide the tail behavior that actually affects users.
• Saturation indicators — CPU, memory, connection pool usage — that predict a problem before it becomes visible in error rate.

A rollout gate that only checks "did the pod crash" catches a small fraction of real production regressions.

Infrastructure changes need the same discipline

Application code isn't the only thing that changes — infrastructure does too. Running terraform plan as a required, reviewed step before apply gives the same benefit for infrastructure that code review gives for application changes: a chance to catch an unintended change (like an accidentally widened security group, or a resource being replaced instead of updated) before it happens.

Rollback has to be automatic, not documented

A rollback runbook is only useful if someone reads it in time. Wiring rollback into the same pipeline that performs the rollout — triggered by the same health signal used to gate progression — means the system reverts a bad release in minutes instead of waiting on a human to notice, diagnose, and manually intervene.

The shape that tends to work

commit → CI (build/test/scan) → terraform plan (validate) →
canary rollout → monitor real traffic → full rollout OR automatic rollback

This progressive-delivery structure is the deployment approach used across the Distributed SaaS Platform system, where automated validation and progressive deployment were primary levers for release safety.