DevOps Daily with Fexingo: CI/CD, Kubernetes, and Modern Software Operations

In this episode of DevOps Daily, hosts Lucas and Luna dive into the hidden dangers of Kubernetes Topology Spread Constraints. They explain how misconfigured spread constraints can prevent pods from scheduling, cause resource fragmentation, and trigger unexpected failovers. Using a real-world example of a three-node cluster with a rollout gone wrong, they walk through the mechanics of how these constraints interact with node affinity and taints, and why the scheduler sometimes makes counterintuitive decisions. They also share practical tips for auditing spread constraints and avoiding common pitfalls. If you manage Kubernetes clusters, this episode will save you from a late-night debugging session.

#Kubernetes #TopologySpreadConstraints #PodScheduling #DevOps #ContainerOrchestration #ClusterManagement #Scheduler #Rollout #HighAvailability #InfrastructureAsCode #TechOps #CloudNative #PlatformEngineering #FexingoBusiness #BusinessPodcast #Technology #SoftwareEngineering #SRE

Keep every episode free: buymeacoffee.com/fexingo

In this episode, Lucas and Luna dig into a nasty failure mode that shows up when ArgoCD's sync phases and hooks are misused: a single application rollout that triggers a cascade of errors, from failed health checks to stuck syncs. They walk through a real-world scenario from a mid-size fintech's production cluster, where a seemingly straightforward update to a payments service brought down dependent services because the sync wave ordering clashed with readiness gates. They explain how ArgoCD's sync waves and resource hooks work, why the default 'sync' phase is often misunderstood, and how a team can end up with a deployment that looks successful in the UI but is actually broken underneath. They share practical guardrails: using sync waves deliberately, matching them with readiness probes, and always testing the sync order in a staging cluster before trusting the production pipeline. If you've ever wondered why your ArgoCD app says 'Synced' but your users are seeing errors, this episode is for you.

#ArgoCD #Kubernetes #GitOps #CICD #DevOps #SyncWaves #SyncHooks #DeploymentFailures #ReadinessProbes #Fintech #ProductionOutages #ContinuousDeployment #InfrastructureAsCode #SoftwareReliability #SiteReliabilityEngineering #CloudNative #FexingoBusiness #BusinessPodcast

Keep every episode free: buymeacoffee.com/fexingo

In this episode of DevOps Daily with Fexingo, hosts Lucas and Luna uncover a quiet but dangerous gap in Kubernetes security: running Pod Security Admission in audit mode can let risky workloads slip into production without a single warning. They trace a real scenario where a team thought they were protected by PSA audit logs, but misconfigured exclusions and permissive policies meant no alert ever fired. Drawing on the Kubernetes 1.23 introduction of Pod Security Standards, Lucas explains how audit mode differs from enforce mode, why relying on logs alone creates blind spots, and how a simple validation check—like a pre-deployment policy scan—can save your cluster from a nasty surprise. Perfect for platform engineers and SREs who think they've got Pod Security covered. Tune in to learn one concrete takeaway: audit mode is a starting point, not a safety net.

#Kubernetes #PodSecurityAdmission #PodSecurityStandards #AuditMode #DevOps #KubernetesSecurity #PodSecurity #PlatformEngineering #CloudNative #KubernetesTips #SecurityBestPractices #SRE #CI/CD #TechPodcast #FexingoBusiness #BusinessPodcast #DevOpsDaily #KubernetesAdmission

Keep every episode free: buymeacoffee.com/fexingo

In this episode of DevOps Daily, Lucas and Luna dig into a silent data-loss hazard in Kubernetes: the PersistentVolume reclaim policy. They walk through a real incident where a Recycle policy deleted a production database's data during a routine Helm upgrade, and explain the differences between Retain, Recycle, and Delete. They cover why the default Delete policy can be dangerous for stateful workloads, how to audit your cluster for risky PVs, and best practices for protecting persistent data with Retain or modern CSI snapshots. If you run stateful applications on Kubernetes, this episode gives you a concrete checklist to avoid an irreversible mistake.

#Kubernetes #PersistentVolume #DataLoss #DevOps #StatefulWorkloads #Helm #CSI #Storage #RetainPolicy #RecyclePolicy #Tech #Business #FexingoBusiness #BusinessPodcast #CloudNative #Infrastructure #SiteReliability #DataProtection

Keep every episode free: buymeacoffee.com/fexingo

In this episode of DevOps Daily, Lucas and Luna dig into a surprisingly common Kubernetes failure: StatefulSets that keep rolling out old images because the controller's ordering logic interacts badly with image pull policies and rolling update strategies. Using a concrete example from a payment-processing service at a European fintech, they explain why 'Always' pull policy can mask a stale image, why the ordinal index (pod-0, pod-1...) creates a false sense of progress, and how a single misconfigured update strategy can leave pods running old code for weeks. They walk through the exact kubectl commands to verify what image each pod is actually running and how to fix the rollout without deleting the StatefulSet. Along the way, they touch on the difference between onDelete and RollingUpdate, the role of revision history, and why you should never trust 'kubectl rollout status' alone. If you've ever scratched your head wondering why your StatefulSet isn't updating, this episode gives you a mental model and a debugging path.

#Kubernetes #StatefulSet #DevOps #CI/CD #RollingUpdate #ImagePullPolicy #Kubectl #PodOrdinal #Fintech #DeploymentStrategies #ContainerOrchestration #TechOps #SoftwareDeployment #CloudNative #K8sTroubleshooting #FexingoBusiness #BusinessPodcast #TechPodcast

Keep every episode free: buymeacoffee.com/fexingo

In this episode, Lucas and Luna dig into a production incident where a service mesh's mutual TLS certificates expired overnight, silently killing traffic between microservices. They walk through the failure mode: certificates that were set to rotate every 24 hours but got stuck because of a clock skew issue on one of the worker nodes. They explain how Kubernetes CertificateSigningRequest objects work, why the rotation loop can fail silently, and what happens when the mesh's control plane doesn't fail open. They also discuss practical mitigations, like monitoring certificate expiry as a first-class signal, setting up alerting on rotation failures, and using short-lived certificates with a well-tested renewal path. The episode includes a concrete checklist for anyone running a service mesh in production.

#Kubernetes #ServiceMesh #mTLS #CertificateRotation #DevOps #SiteReliability #ProductionIncident #Microservices #Istio #Linkerd #KubernetesCertificates #TLS #ClockSkew #Monitoring #Alerting #FexingoBusiness #BusinessPodcast #TechOps

Keep every episode free: buymeacoffee.com/fexingo

In this episode, Lucas and Luna dig into a surprisingly common Kubernetes failure mode: when Pod PriorityClasses and preemption interact badly with cluster autoscaling, nodes get drained, workloads get killed, and the cluster autoscaler ends up fighting itself. They walk through a real-world scenario where a high-priority batch job preempts a low-priority web service, the node autoscaler scales down the now-idle node, and the preempted pods can't reschedule because the cluster is too small — creating a deadlock. Lucas explains how preemption is designed to work, why the autoscaler's node utilization threshold can trigger a scale-down right after preemption, and what signals you can look for in events and metrics to catch this before it bites. They also discuss practical mitigations: using PodDisruptionBudgets with preemption, tuning the autoscaler's scale-down utilization threshold, and considering whether preemption is even the right tool for your workload. If you run a cluster with mixed priorities and autoscaling, this episode gives you a concrete failure pattern to recognise and avoid.

#Kubernetes #PodPriority #Preemption #ClusterAutoscaling #DevOps #CloudNative #SRE #PodDisruptionBudget #KubernetesOperators #InfrastructureAsCode #Tech #FexingoBusiness #BusinessPodcast #DevOpsDaily #KubernetesTroubleshooting #Autoscaling #PriorityClass #KubernetesBestPractices

Keep every episode free: buymeacoffee.com/fexingo

In this episode of DevOps Daily, Lucas and Luna explore a growing pain in Kubernetes clusters upgraded to version 1.24 and later: default ServiceAccount token expiration. They walk through the surprising moment a listener's CI pipeline stopped deploying, trace the root cause to a one-hour token time-to-live, and explain how to migrate to projected volume mounts with sidecar refresh. With practical before-and-after YAML and a warning about legacy SDKs that still assume permanent tokens, this is a timely survival guide for anyone running modern Kubernetes. Also, Lucas ties in a broader theme of deprecation velocity in cloud-native, and Luna pushes back on whether the change was disruptive by design.

#Kubernetes #DevOps #ServiceAccount #TokenExpiration #CI-CD #CloudNative #Kubernetes124 #Security #Kubelet #ProjectedVolume #Technology #PodIdentity #Automation #SRE #FexingoBusiness #BusinessPodcast #DevOpsDaily #Deprecation

Keep every episode free: buymeacoffee.com/fexingo

etcd compaction is a routine maintenance operation that removes historical key-value versions, but when it runs, it can consume significant IO and CPU, leading to increased latency for API server requests. This episode dives into the mechanics: compaction triggers defragmentation of boltdb pages, causing pauses that can spike request latency from milliseconds to seconds. We discuss a real-world incident where a 500-node cluster saw API latency jump to 8 seconds during peak hours due to compaction misconfiguration. Then we cover mitigation strategies: tuning auto-compaction retention, scheduling defrag during low traffic, monitoring etcd metrics, and setting alerts for compaction duration. Understanding this often-overlooked performance pitfall is crucial for maintaining responsive clusters at scale.

#Kubernetes #etcd #APIServer #Latency #Compaction #Defragmentation #Performance #Tuning #SRE #DevOps #Infrastructure #CloudNative #FexingoBusiness #BusinessPodcast #Technology #Operations #Cluster #Scalability

Keep every episode free: buymeacoffee.com/fexingo

In this episode of DevOps Daily, Lucas and Luna investigate a production outage triggered by Helm upgrade failure after Kubernetes ConfigMaps were set to immutable. They trace the root cause to a Helm chart that marked ConfigMaps immutable, preventing updates without manual deletion. The hosts explain how this feature, intended for performance optimization, can silently break deployments, and share a concrete workaround using a new ConfigMap name per revision. They also discuss how to audit existing deployments for the immutability flag and when it makes sense to use it.

#Kubernetes #Helm #ConfigMap #ImmutableConfigMap #DevOps #CI/CD #DeploymentFailure #HelmUpgrade #KubernetesTroubleshooting #Prometheus #ConfigurationManagement #DevOpsDaily #Fexingo #Technology #KubernetesConfig #HelmChart #Infrastructure #SRE

Keep every episode free: buymeacoffee.com/fexingo