Information does exist; however, the information is distributed across various documentation formats such as design documents, runbooks, architectural notes, deployment guides, and incident reports. In general, a developer is aware of which services depend on each other (the Checkout Service depends upon the Payment API), the database or technology stack being used by the dependent services (the Payment API utilizes PostgreSQL), and who owns/operates the dependent service (Platform Team owns and operates the Payment API), however, these pieces of information typically reside in separate locations.

Several years ago, my team made a decision that felt obviously correct: If a downstream call fails, retry it. More retries, more resilience. We set three retries on every integration touching our order-fulfillment platform, shipped it on a Thursday, and went home feeling good about our reliability posture.

Six weeks later, retries were the single largest source of traffic in the platform.

An agent does not just call an API. It decides which APIs to call, in what order, and often keeps going until it reaches a result. That autonomy is the point of using an agent, but it is also what makes it dangerous: a single misconfigured agent can generate thousands of requests in minutes, reach systems it was never meant to touch, or chain APIs together in a sequence no human ever designed.

When I needed to validate project intake forms against 60+ business rules of financial, compliance, and other types of business rules and guidelines (some of them to be used in a deterministic way and others to be used in a more nuanced manner), I knew that a simple if-else logic-based manual review process would not scale.

Replacing that manual work with a synchronization service sounds like a junior-level task: read from system A, write to system B, schedule it, done. In practice, sync services are where many integration projects quietly fail. They fail not because moving data is hard, but because the edge cases are partial failures, retries that duplicate records, two systems that both think they own a field, and errors that nobody notices for three weeks.

Spring AI comes with solid support for building RAG systems. It has been almost three years since Spring AI showed up, and in that time it has grown from an experimental member of the Spring portfolio into a mature layer over chat models, embedding models, vector stores, and the plumbing that sits between them, which happen to be exactly the pieces a RAG system needs.

Back in 2019, I inherited a service in exactly this position. It was a .NET-based SOAP web service acting as a middleware layer: a caller would hit our service, our service would reach out to a customer's web service, retrieve the data, and hand a response back to the original caller. Request and response payloads were transformed using XSLT, with a distinct transformation mapped to each customer's expected schema.

I wanted to see how far I could automate it with a small AI agent built using Codex, SerpApi, and Lark.

However, adopting microservices is not simply a matter of breaking a large application into smaller services.

Here’s what is actually shifting, and what stays the same.

That gap between knowing testing matters and actually having good test coverage is where most of the interesting problems live. And a significant part of why that gap exists comes down to framework choices made for the wrong reasons, or made without enough information about what different frameworks actually do well.

Which raises an uncomfortable question for anyone working on a new architecture: if AI is this fluent in 2020-era patterns, are we as an industry going to stay locked into those patterns simply because that's what the model knows? Is AI a conservative force that quietly drags software architecture backward to its training data's center of mass, no matter how good a newer idea might be?

The Pitch Everyone's Making Right Now

Spec-driven development has become the default answer to "AI agents write plausible-looking code that's subtly wrong." The idea: stop prompting, write a structured spec, let the agent execute against it, review the diff. GitHub Spec Kit has 90K+ stars and /speckit.specify → /speckit.plan → /speckit.tasks → /speckit.implement is basically a known workflow at this point. OpenSpec does the same thing lighter, with delta specs and openspec validate --strict. BMAD goes the other direction and simulates a full 12-agent team. AWS built an entire IDE (Kiro) around the idea. Tessl raised $125M on it.

Continuous API evolution is non-negotiable in contemporary software development, yet maintaining backward compatibility remains an incredibly expensive and labor-intensive hurdle. Core schema mutations frequently force downstream enterprise clients into disruptive and unplanned refactoring cycles, stalling product velocity.  

The typical industry fix — maintaining multiple, hard-coded API routes (e.g., /v1, /v2) — inevitably results in severe codebase sprawl, fractured engineering focus, and massive technical debt for the API provider.  

By the end, you'll have a Streamlit app with three tabs, one for news articles, one for YouTube videos, and one for social media and forum discussions. We'll use "serpapi" as the search query, but you can swap the brand or product name. 

Picture this. Your team has just integrated a large language model into your enterprise application. The demo looked compelling. The agent interpreted user intent, called several APIs, and returned a coherent result. Everyone in the room was impressed.

Then the questions started. What happens when the LLM misinterprets a request and calls the wrong API? Who owns the business logic embedded in that prompt? If the model changes, does the integration break? How do you audit what the AI decided and why?

Introduction

Before diving into the programmatic details, first some concepts need to be explained. Do realize that in this blog, only the surface is scratched from what is possible with RabbitMQ. A detailed overview can be found in the official RabbitMQ documentation.

There's an idea making the rounds in B2B SaaS product and engineering meetings right now. It sounds reasonable. It feels optimistic. And it's leading companies straight into the same trap they've always fallen into, just at an accelerated rate.

Despite this, API design has long faced a notable gap.

Five Key Takeaways

1. When your OpenAPI specification becomes the single source of truth, the gap between your API contract and your gateway configuration simply stops existing.
2. Generating the gateway from the spec scales far better than hand-maintaining per-endpoint configuration as your API surface grows into the hundreds.
3. Generated, human-readable service code keeps day-to-day operations manageable — you can read it, reason about it, and trace failures like ordinary software.
4. The genuinely hard part is not the generation; it's the regeneration workflow and the discipline around where custom logic is allowed to live.
5. Adopt the model on new APIs first, prove it's boring and trustworthy, and only then migrate existing ones.

The Quiet Way Gateways Rot

Every public API gateway I've worked with started its life clean and, over a few years, quietly accumulated a second universe of hand-written configuration sitting alongside the services it fronts. None of it looked dangerous at the time. A path rewrite here. A parameter rename there. A response transform to make an internal field look the way customers expect it to. A content-type translation to bridge two teams that made different choices years apart. Each individual edit was sensible, small, and well-intentioned. The danger was never any single change — it was the accumulation, and more importantly, the separation.

That configuration described how the gateway should behave, but it lived in a different place from the thing it was describing: the API's actual contract. Two artifacts, two repositories, two owners, two review processes, two release cadences — all trying to stay in agreement about the same set of endpoints. Anyone who has run a system like this knows how that story ends. The two drift apart. A backend team renames a field and ships their service. The matching gateway mapping doesn't get updated because it's someone else's pull request in someone else's repo. Nothing fails loudly. A customer-facing response is simply, silently wrong. And the place you now have to go and debug is the gateway — the one component that every single request flows through, and therefore the one component nobody wants to touch under pressure.