Note the difference between fully autonomous agents like Claude Code or Codex, and chat-based agents that still react to user input. This post is about the latter.

most Retrieval-Augmented Generation (RAG) systems in production today are fragile, expensive, and deceptively incomplete.

The Identity Model We Built Is Already Broken

Traditional IAM, PAM, and SSO tools were designed for a world where actions map cleanly to people. An employee logs in, performs tasks, logs out. Audit trails are straightforward. Authorization decisions are binary.

While a significant amount of security effort still centers on production defenses, the most effective attacks are increasingly targeting upstream, where artifacts are created and dependencies are resolved. And one of the most preventable (yet still common) entry points is also one of the earliest: secrets leaking into source control.

Here’s the thing — the conversation that’s not happening enough in health plan architecture reviews isn’t about the next HIPAA update or even about migrating to the cloud. It’s about the massive, hidden opportunity cost of treating EDI as just another compliance checkbox. While we’ve perfected transaction processing to an art form, we’ve accidentally locked away our industry’s most valuable operational data in what amounts to digital silos. Look, I get it — if it isn’t broken, don’t fix it. But what if “working” isn’t good enough anymore? The real need right now isn’t another SpecBuilder tweak or version upgrade; it’s a complete mindset shift from seeing EDI as a cost center to treating it as your primary, living, breathing strategic data asset.

Reliability engineering has historically relied on a predictable workflow. A monitoring system detects an anomaly, an alert is triggered, and an engineer investigates logs and metrics before applying a remediation step. This model works reasonably well for traditional applications where failures occur slowly and are relatively easy to diagnose. AI-driven systems behave differently.

Modern AI platforms are built on layers of interconnected services. A typical architecture may include data ingestion pipelines, feature generation systems, vector databases, inference services, and orchestration frameworks that coordinate agents or downstream automation workflows. Failures rarely occur in isolation. A minor delay in a retrieval service can increase inference latency, which then cascades into application-level instability. In high-throughput systems processing thousands of requests per minute, such instability can propagate across the entire system before engineers have time to investigate the initial alert.

AI has hit the gas pedal on software delivery. We are shipping more code, more often, and relying on automated logic and external dependencies, which expands the attack surface beyond what existing practices were designed to catch.

In the era of Artificial Intelligence, AI systems are used to make decisive systems in healthcare, insurance, finance, hiring and customer engagement domains. In these domains if the system fails it will directly impact cost or loss of trust on the system.

To maintain the failure of AI systems. Most of the organizations introduce Responsible AI policies, ethical principles and governance frameworks. After all these efforts, AI system incidents is happening and in increasing trend.

I have seen this pattern more than once. A team builds an agent that summarizes tickets, queries CRM data, and opens service requests. The demo lands well. Leadership wants it in production next month. The agent works, but production is not a quality bar; it is an operational contract. The moment an agent can call a tool, it stops being an ML artifact and becomes production software.

Most of what we know about shipping production software still applies: identity, authorization, logs, rate limits, and rollback. None of this is new. But four assumptions of traditional ops quietly break when the caller is an agent. Execution is no longer deterministic. An HTTP 200 no longer means the action was correct. The threat surface is not static; it grows with every prompt. And on-call engineers cannot resolve every incident on their own, because the relevant judgment is often a business one.

Just like POST requests, the content-type header is important because it tells the server how to interpret the data we’re sending.

Modern infrastructure has evolved much faster than the way we operate it.

Today’s systems are distributed, constantly changing, and deeply interconnected. A single user request can move through many services, each producing logs, metrics, and traces. We now have more visibility than ever before.

In this article, instead of revisiting those well-known concepts, vLLM (v0.20.0) KV cache implementation details are discussed for a deeper understanding. By walking through code internals with concrete code pointers and design insights, the goal is to bridge the gap between high-level understanding and real-world system design.

We all have that one folder. The one labeled "v1_final_do_not_touch_2016." It is a sprawling ecosystem of spaghetti code, global variables, and comments that simply read // I am sorry. In an era of large language models (LLMs), we often hear about AI writing boilerplate, but can it actually perform digital archeology?

I decided to feed my most "haunted" legacy script — a 2,000-line monolith responsible for processing data — into a hypothetical next-generation model, Gemini 3. The goal wasn't just to see if it could fix the bugs, but to see if it could transform a maintenance nightmare into a modern, scalable architecture.

This article covers the four frameworks I have personally used to ship Java AI applications: Genkit Java, Spring AI, LangChain4j, and Google ADK Java. Each one represents a meaningfully different bet on what a Java AI framework should be, and understanding those differences will save you from picking the wrong tool.

1. Knowledge workers and other content producers – how can we survive the AI wave with some kind of defensible capability we can offer our employers and our audiences that AI won’t be able to replace, even as it matures?
2. Software vendors – how can we survive the AI wave with some kind of defensible product capability we can offer our customers that AI agents won’t be able to replace, even as they mature?

If you’re a pessimist, the situation may seem hopeless. AI is getting so much better so quickly that even if it can’t quite replace us or our software products today, it’s only a matter of time, right? Should we abandon hope?

This is what I mean when I call uncontrolled retries a self-inflicted Denial-of-Wallet attack. Not metaphorically. Mechanically.

I recently participated in a makeathon focused on building AI-powered applications. Over 2–3 intense days, I watched teams go from idea to demo — and the patterns that separated working products from frustrated debugging sessions were remarkably consistent, especially for teams building AI agents.

From this makeathon and from my experience working with teams building AI applications and agents, here are the three lessons I took away on how to build reliable AI applications by engineering around non-determinism. Together, these form what I would like to call “The Architecture for Reasoning Control”.

From a surface-level perspective, agentic systems are a new abstraction. But if we look under the hood, we find something that looks rather familiar: distributed systems.

Yet the outputs are wrong. In many AI deployments, engineers eventually encounter this situation: a recommendation system begins suggesting irrelevant items, a support chatbot produces inconsistent answers, or an AI assistant gradually becomes less reliable in answering domain-specific questions.