Sponsored By: Nutanix The following is sponsored content. It may not reflect the views of our editorial staff.

The Kubernetes scaling problem nobody talks about

Enterprise platform teams encounter the same pattern repeatedly: a Kubernetes platform works well enough that nobody wants to change it.

This happens gradually as teams make reasonable technology choices: selecting different ingress controllers, secrets management tools, CD platforms, or observability software. Individually, none of these decisions is a problem. Months later, however, they’ve created a Kubernetes environment that only a handful of people understand. As soon as that one person gets sick or leaves the company, maintaining or improving the platform becomes much more difficult.

It covers:

Why This Matters

Traditional microservices assume that applications already know which services to invoke. An Order Service calls Inventory, Payment, and Shipping because the workflow is explicitly encoded during development. An AI agent, however, begins with an intent rather than an execution path. A request such as "purchase the least expensive laptop available and deliver it tomorrow" requires evaluating inventory, pricing, promotions, shipping constraints, and fraud policies before any API is called. The workflow is determined during execution instead of implementation.

The Business Case: The Friction of the "Comfort Gap"

I have three primary alter egos. Early in the mornings, I teach Spanish. Nothing fancy, just a simple, online session, focused on one student at a time, sharing and imparting what I learned and how I learned, to help them benefit from knowing Spanish as their second language. The rest of the day is spent in my Enterprise Architecture work — from consulting, to product development, to strategic solutions, and you know… all the standard corporate jargon. And then late at night, I imagine mysteries and write fiction.

All that is fine. But then one of the most awkward conversations I have to have occasionally is telling my student: “Hey, so… you’ve used 10 classes and only paid for 10 classes… physics dictates we cannot proceed without a renewal.” Awkward, right? One morning where I needed to have that exact conversation, I thought to myself, “Ha! Let me hire an operations manager to handle these. I just need to see the details on the Kanban board later.”

Building agentic AI systems fundamentally changes how we handle application security. We are no longer just securing our own code. We are securing our infrastructure against code written dynamically by an LLM and executed on the fly. When building a multi-tenant AI platform, allowing an agent to run arbitrary scripts is a massive escape vector waiting to happen.

Google recently made the GKE Agent Sandbox generally available on their custom Arm-based Axion N4A instances. This gives us a highly efficient, hardware-optimized path to run untrusted code safely. Under the hood, this relies on gVisor to intercept application kernel calls and run them in a heavily restricted user-space kernel.

Nobody on the infra side had touched Docker that day. The incident wasn't a Docker bug. We assumed that container orchestration designed for web services would function the same way for processes that take minutes to become useful, rather than those that operate in milliseconds.

When I needed to validate project intake forms against 60+ business rules of financial, compliance, and other types of business rules and guidelines (some of them to be used in a deterministic way and others to be used in a more nuanced manner), I knew that a simple if-else logic-based manual review process would not scale.

Nothing in the logic had changed. The environment was the only obvious difference. That was the day I stopped treating the runtime environment as an afterthought and started treating reproducibility as part of the experiment itself.

The root cause was painfully simple: Spring Boot needed 45 seconds to start, and my ALB health‑check timeout was 5 seconds. The tasks never had a chance.

Spring AI comes with solid support for building RAG systems. It has been almost three years since Spring AI showed up, and in that time it has grown from an experimental member of the Spring portfolio into a mature layer over chat models, embedding models, vector stores, and the plumbing that sits between them, which happen to be exactly the pieces a RAG system needs.

You send a prompt to a model, get a response back, and wire it into your app. Done.

While cloud-based AI assistants offer impressive capabilities, they often require subscriptions and sharing potentially sensitive application data with third-party services.

That night is the reason I now have strong opinions about Docker and AI infrastructure.

Sponsored By: Nutanix The following is sponsored content. It may not reflect the views of our editorial staff.

Most platform engineers are kept awake at night with some form of the same common complaint: the infrastructure bill does not align with what the infrastructure is really doing.

For example, a GPU node pool provisioned for a monthly batch job might sit idle, burning budget for 20+ days out of 30. Or perhaps a business builds a standby data center designed specifically to account for a potential major outage, but that sits idle doing nothing every other day. CI/CD runners wait listlessly for the next pipeline trigger: fully provisioned, fully billed, but mostly idle.

However, adopting microservices is not simply a matter of breaking a large application into smaller services.

This article is a reference implementation. JWT authentication done correctly, rate limiting that survives distributed deployments, input validation that catches more than annotations alone, and output encoding that doesn’t break under the edge cases. Each section has the code that should be on every API by default, with the rationale for why.

This code smell is more than just an eyesore; it creates significant technical debt. It is exceptionally difficult to unit test, violates fundamental object-oriented design principles, and introduces severe regression risks where adding a single business rule threatens to break three existing ones.

Which raises an uncomfortable question for anyone working on a new architecture: if AI is this fluent in 2020-era patterns, are we as an industry going to stay locked into those patterns simply because that's what the model knows? Is AI a conservative force that quietly drags software architecture backward to its training data's center of mass, no matter how good a newer idea might be?

This challenge becomes more visible in microservice-based systems where repositories, services, and automation suites are distributed across multiple projects. A small change in one module can unintentionally trigger an entire regression pipeline containing tests unrelated to the updated code.

Modern enterprise software design has fundamentally shifted away from monolithic, single-threaded runtimes toward decoupled, containerized architectures. When building systems that handle high throughput — such as fintech services, automated reporting pipelines, or real-time distributed platforms — engineers must address two core infrastructure vectors: high-concurrency connection management and deterministic relational state execution.

A common anti-pattern in backend systems engineering is assuming that containerization automatically scales an application. In reality, wrapping a poorly optimized, blocking database service inside a Docker container simply shifts the performance bottleneck from local computing hardware to network sockets and thread pools.