DevOps Daily with Fexingo: CI/CD, Kubernetes, and Modern Software Operations

In this episode of DevOps Daily with Fexingo, Lucas and Luna dive into a specific failure mode that often surprises even experienced Kubernetes operators: cascading failures caused by mishandled node draining. They walk through a real-world scenario where a routine node drain for a kernel patch triggered a chain reaction: PodDisruptionBudgets were respected on paper, but the remaining nodes couldn't handle the load, leading to resource starvation and a partial cluster outage. They break down why the Kubernetes scheduler doesn't account for combined resource pressure during draining, how default eviction timeouts can mask the problem, and what operational practices — like pre-drain load testing and gradual cordoning — can prevent the cascade. No fluff, just one concrete angle with actionable takeaways.

#Kubernetes #NodeDraining #CascadingFailures #PodDisruptionBudget #ClusterResilience #SRE #DevOps #CloudNative #Operations #ProductionIncident #KubernetesScheduler #EvictionTimeout #LoadTesting #HighAvailability #Infrastructure #Technology #FexingoBusiness #BusinessPodcast

Keep every episode free: buymeacoffee.com/fexingo

In this episode, Lucas and Luna dive into a deceptively tricky Kubernetes feature: CronJob timezone handling. They break down how CronJobs default to UTC regardless of the cluster's regional settings, leading to off-schedule jobs, missed backups, and batch processing failures. Using a real-world example from a fintech company that lost two hours of payment reconciliation data, they explain why the .spec.schedule timezone field added in Kubernetes 1.27 is still underused, and why operators relying on TZ environment variables are playing with fire. They also cover how kube-controller-manager's timezone drift can compound the problem, and share a simple audit command to check all CronJobs for implicit UTC assumptions. If you manage scheduled jobs in Kubernetes, this episode will save you from a midnight pager.

#Kubernetes #CronJob #Timezone #DevOps #CloudNative #Scheduling #ProductionBug #Fintech #BatchProcessing #KubeControllerManager #UTC #SiteReliability #PlatformEngineering #TechPodcast #FexingoBusiness #BusinessPodcast #DevOpsDaily #SoftwareOperations

Keep every episode free: buymeacoffee.com/fexingo

Lucas and Luna revisit pod security standards in Kubernetes, digging into a specific case where a restricted PSP still allowed privilege escalation through a misconfigured seccomp profile. They walk through a real-world example from a fintech startup that ran a compliance audit and discovered their 'secure' pods were running with default-deny seccomp disabled. The conversation covers why PSPs and PSS profiles are not a silver bullet, how admission controllers can be bypassed, and what operators should actually check in their cluster logs. No fluff, just the concrete gap between policy intent and runtime reality.

#Kubernetes #PodSecurity #Seccomp #ContainerSecurity #DevOps #Technology #CloudNative #SecurityAudit #PSP #PSS #AdmissionController #RuntimeSecurity #Fintech #Compliance #FexingoBusiness #BusinessPodcast #Podcast #DevOpsDaily

Keep every episode free: buymeacoffee.com/fexingo

Lucas and Luna break down a subtle but costly Kubernetes anti-pattern: using pod priority and preemption without proper resource accounting. They walk through how a mid-stage startup accidentally triggered a cascading preemption storm by misconfiguring priority classes on their CI/CD runners and ML training pods, leading to 40% wasted compute and unpredictable evictions. Lucas explains the math behind the scheduler's preemption logic, why lower-priority pods can still cause cluster thrashing, and three concrete fixes: bin packing via node selectors, using descheduler plugins, and setting priority thresholds based on actual workload criticality. Luna pushes back on the 'just set higher priority' mindset and asks whether priority classes even make sense without guaranteed QoS. The episode includes a natural donation segment around the 25% mark where the hosts reflect on the value of ad-free content. Perfect for anyone managing Kubernetes clusters who has ever wondered why their nodes look full but their applications feel empty.

#Kubernetes #PodPriority #Preemption #ClusterScheduler #ResourceManagement #CloudNative #DevOps #K8sAntiPatterns #NodePressure #BinPacking #Descheduler #QualityOfService #CICD #MLTraining #ClusterThrashing #FexingoBusiness #BusinessPodcast #Technology

Keep every episode free: buymeacoffee.com/fexingo

Episode 45 of DevOps Daily with Fexingo digs into Kubernetes topology spread constraints – the feature meant to spread pods evenly across failure domains, but that can actually create severe node imbalances. Lucas and Luna explore a real incident where a three-node cluster ended up with 40 pods on one node and 4 each on the others, causing resource exhaustion and cascading failures. They explain how 'maxSkew' and 'whenUnsatisfiable' interact in surprising ways, why topology spread constraints don't play well with autoscalers, and how to avoid the 'spread paradox' where enforcing balance creates imbalance. Tune in for practical configuration advice and a breakdown of when to use topology spread instead of pod affinity.

#Kubernetes #TopologySpreadConstraints #MaxSkew #PodScheduling #DevOps #K8s #ClusterBalancing #NodeImbalance #PodAffinity #CloudNative #FexingoBusiness #BusinessPodcast #Technology #CI/CD #SoftwareOperations #Scheduling #Autoscaler #KubernetesFailure

Keep every episode free: buymeacoffee.com/fexingo

Lucas and Luna dive into a hidden cost driver in Kubernetes clusters: node priority and the scheduler's 'most requested' scoring. They explain how default scheduling behavior over-provisions nodes, why even well-tuned clusters run 15-30% extra capacity, and a concrete fix using pod priority classes and node affinity. The episode walks through a real-world case where a mid-size fintech reduced its GKE bill by 22% by adjusting priority scoring weights. A must for DevOps engineers wondering why their cloud spend keeps creeping up despite right-sizing.

#Kubernetes #DevOps #CloudCost #NodePriority #Scheduler #ClusterAutoscaling #GKE #PodPriorityClass #NodeAffinity #CostOptimization #CloudNative #FinOps #K8s #Infrastructure #Technology #FexingoBusiness #BusinessPodcast #DevOpsDaily

Keep every episode free: buymeacoffee.com/fexingo

Episode 43 of DevOps Daily with Fexingo dives into a hidden failure mode of the Kubernetes Cluster Autoscaler: it frequently fails to scale up new nodes quickly enough after a spot instance interruption. Lucas explains how the default unready-node-taint strategy can delay scale-up by several minutes, causing pods to stay pending and potentially triggering cascading failures. He walks through a real incident at a mid-size SaaS company where a 40% spot interruption rate led to 15-minute scale-up latencies and broken SLOs. Luna challenges whether unscheduled pods on a tainted node really need to be recreated from scratch, and the pair discuss workarounds like increasing the scale-up ratio of the CA's expansion options, using the new PodTopologySpread constraints, and pre-provisioning a small buffer of on-demand nodes. The episode also covers upcoming Kubernetes 1.32 changes to the Cluster Autoscaler's handling of instance interruptions. No ad breaks, ever. If today's tech conversation gave you something usable, consider supporting the show at buy me a coffee dot com slash fexingo.

#Kubernetes #ClusterAutoscaler #SpotInstances #DevOps #CloudNative #SRE #K8sScaling #PodDisruption #InstanceInterruption #NodePool #Autoscaling #K8sFailureMode #CloudEconomics #SaaSInfrastructure #PodScheduling #K8s1.32 #Technology #FexingoBusiness

Keep every episode free: buymeacoffee.com/fexingo

Episode 42 of DevOps Daily with Fexingo digs into a concrete problem with Kubernetes Horizontal Pod Autoscaler (HPA): it reacts to past metrics, not future demand. Lucas and Luna walk through a real-world scenario where a flash sale at an e-commerce company caused a five-minute latency spike before HPA caught up. They explain how the default two-minute cooldown window, aggregation intervals, and metric collection delays compound into a blind spot for bursty traffic. The conversation covers tweaks like reducing stabilization windows, using custom metrics tied to queue depth, and combining HPA with cluster autoscaler or predictive scaling via Vertical Pod Autoscaler and proactive budget-based approaches. No silver bullets, but a clear diagnosis: if your traffic comes in waves, default HPA settings will leave you exposed. Specific numbers, trade-offs, and one actionable recommendation per tuning strategy.

#Kubernetes #HorizontalPodAutoscaler #HPA #CloudNative #DevOps #Scaling #Autoscaling #Latency #MetricsServer #CustomMetrics #KEDA #ClusterAutoscaler #BurstTraffic #Performance #Technology #FexingoBusiness #BusinessPodcast #DevOpsDaily

Keep every episode free: buymeacoffee.com/fexingo

Episode 41 of DevOps Daily with Fexingo dives into the overlooked security risks of Kubernetes Ingress controllers. Lucas and Luna break down a real-world case where a misconfigured NGINX Ingress exposed internal APIs to the internet, leading to a data breach. They explain how default rules, lack of request validation, and forgotten annotation changes can turn a routing layer into an attack vector. The hosts also discuss practical mitigations: strict allowlists, Web Application Firewall integration, and annotating Ingresses for least-privilege access. If you're running Kubernetes in production, this episode shows why your Ingress controller might be your biggest blind spot.

#Kubernetes #IngressController #NGINXIngress #SecuritySurface #DevOps #CloudNative #K8sSecurity #NetworkSecurity #APISecurity #WebApplicationFirewall #AccessControl #TechPodcast #FexingoBusiness #BusinessPodcast #DevOpsDaily #SoftwareOperations #CI/CD #Microservices

Keep every episode free: buymeacoffee.com/fexingo

In this episode of DevOps Daily with Fexingo, Lucas and Luna dive into a commonly overlooked vulnerability in Kubernetes clusters: ConfigMaps. While ConfigMaps are widely used to decouple configuration from container images, they often end up storing sensitive data like database passwords, API keys, and service tokens in plain text. This practice creates a security liability that can be exposed through simple misconfigurations, RBAC gaps, or even accidental commits to version control. The hosts walk through a real-world scenario from a mid-sized fintech that learned this the hard way when a developer's CI/CD pipeline accidentally pushed a ConfigMap containing production database credentials to a public GitHub repo. They explain how to audit your ConfigMaps for secrets, why you should never use them for sensitive data, and how tools like Sealed Secrets and external secret stores (like HashiCorp Vault or AWS Secrets Manager) can replace ConfigMaps for secret management. The episode closes with a practical checklist to secure your configuration data today.

#Kubernetes #ConfigMap #Security #DevOps #SecretsManagement #K8sSecurity #CI/CD #Technology #Podcast #FexingoBusiness #BusinessPodcast #TechPodcast #CloudNative #DevSecOps #Vault #SealedSecrets #KubernetesSecurity #ConfigurationManagement

Keep every episode free: buymeacoffee.com/fexingo