Corentin Sueur

Field notes — 03

Mocking the world, one provider at a time.

Field note 01 ended with a test-environment API that owns state. This is its third act: the same service now stands in front of every third-party provider our integration environments talk to, and decides per request whether to answer from a rule, forward to the real thing, call us back later, or let a running test answer live. Here is what a mock has to get right before a team trusts it.

The problem

Note 01 drew a hard line: no mocking the world. The suite's value is that dev and stage run as deployed, third parties included. Two years of living with that line taught me where it actually belongs. The services, the queues and the databases must stay real. The HTTP call that leaves the cluster for a top-up provider, a messaging platform or a fraud-scoring service is a different matter.

Those calls cost real sandbox credit on every test run. Provider sandboxes went down and took our pipeline with them, and some behaved unlike production: one reported a state that never completes, so the happy path could not even be exercised. And a test could not make a provider misbehave on purpose. "What happens when the provider declines" was the most important scenario we had, and the one we could not run.

The reframing was the same as in note 01, one layer further out: the edge where our traffic leaves the cluster is a surface, and nobody owned it. Own it, and the decision to mock or forward stops being a property of an environment and becomes a property of a request.

Constraints

  • Per request, not per environment. The same environment must forward one call to the real provider and answer the next from a rule, with nothing redeployed in between.
  • The apps change one line. A provider's base URL. No test hooks in production code, no "if testing" branches.
  • A test can make the provider do anything. Decline, time out, answer with a specific body, and report the outcome later through a callback, the way the real one does.
  • A mock leaves evidence. What was answered, by which rule, for which run. A reply nobody can trace is a reply nobody can debug.
  • Shared environment, isolated runs. A rule one test arms must not surprise another test, and must not outlive its run.

The design

One port per provider, one decision per call

Each provider gets its own port on the tool, and the app under test is configured to call that port instead of the provider. Thirteen providers sit behind it today, five of them fully mocked. Every call then climbs the same ladder: a connected test gets the first chance to answer, then the first matching rule, then a reply built in for that provider, then the provider's default reply. When a port is armed and nothing matches, the answer is a loud 500 that says so, never a quiet forward to the real provider. A silent fallback is how a test suite starts spending money again without anyone noticing.

live test handlerfirst matching rulebuilt-in replydefault reply500 — armed, no rulemost specific — the test speakscatch-all — then fail loudlyper run — the test decides
The resolution ladder. The two left rungs belong to the running test; the rest belong to whoever maintains that provider's rules.

Rules are expressions over the request: method, path, headers, body. They run on Microsoft RulesEngine, one workflow per provider, first match wins. Replies are Handlebars templates that can read the request, so a mocked order confirmation echoes the amount and the id it was asked about. A body without a template marker is served as bytes, which matters more than it sounds; see below.

How a request is answered

The part that surprised people most is the first rung. A system test can open a long-lived stream to the tool and answer intercepted calls itself, in test code, with the test's own variables in scope. No rule to write, no state to clean up: when the test ends, the stream closes and the rung disappears. For everything the test does not want to handle, it says skip and the ladder continues.

app under testinterception toolsystem testreal providercall on the provider's portoffered to a connected test firstanswer — or skiprule → built-in → default replyforward only if a rule says sostreamed backcallback queued before the reply leavesmocked reply, Content-Length setrecorded: by port, by rule, by run
One intercepted call, start to finish. Forwarding is a decision a rule makes, not a default the tool falls back to.

Rules belong to runs

Hand-written rules live as a seed file in git, can be edited in the browser with a hundred snapshots of history, and are loaded into the in-memory engine at startup. The interesting rules are the other kind. A test, or an agent, arms a rule for its run: the rule is appended to the provider's list, named for the run, and registered as a tracked entity of that run token. Appended, not inserted, so a hand-written rule that matches the same call still wins and the tool tells you how many stand ahead of yours. And because it is a tracked entity, the teardown from note 01 removes it, whether the run ends politely or the runner dies. The two notes are the same design, one layer apart.

seed in giteditor, 100 snapshotsengine in memorySQL, within 30 sarmed rules — appended per run, named for the runteardown of the run removes its armed rules — the same teardown as note 01hand-written rules sit first in the list, so an armed rule can never override one
Where rules come from and where they go. The run token that owns fixtures owns rules too.

The provider that calls you back

Some providers report the outcome later, by calling us: the top-up completed, declined, reversed. A mock that only answers the request is mocking half the provider. So a rule can carry a callback: a template, a delay, and the values to fill in. The ordering is the whole trick. The callback is queued before the mocked reply leaves, so a client that immediately polls the status with the id it just received finds a pending callback and gets "pending" back, exactly as it would from the real provider. Then the callback fires at our service, through the same public address it would use in production, hosts allow-listed, no redirects, short timeout.

provider callcallback queuedmocked replystatus poll: pendingcallback firesbefore the reply leaves — so a poll finds itpending → sending → sent | failedno silent retry — a failed callback stays visible; replaying by hand sends a copyallowed hosts only · no redirects · 15 s timeout
A callback's life. Queue first, reply second, or the poll that follows the reply has nothing to find.

A failed callback does not retry on its own. It stays visible as failed, with what was sent and what came back, and a human or an agent can replay it, which sends a copy. The lesson from note 01 again: a failure you can see beats a clean-looking lie, and an automatic retry against a test environment is a very good way to manufacture one.

Evidence, kept twice

Every intercepted exchange is recorded for two readers with different needs. A human debugging now wants the last few hundred calls on this port, bodies included, and wants them this second: that is an in-memory ring per port. A test or an agent wants "everything my run caused", days later, by run token: that is a table in SQL, partitioned by month, pruned on a schedule. Plain forwards to real providers are stored without bodies, since no run can claim them and they are most of the volume. The attribution rule is strict on purpose: a call answered by a hand-written rule is never credited to a run, only calls answered by rules the run armed.

in memory — for a human looking nowQA in a browser500 per port60 minutesfull bodiesring bufferin SQL — for tests and agentssystem testsAI agentsby run tokenone monthforwards: no bodypartitions
Two stores, two readers. The ring is for looking; the partitions are for proving.

Agents at the same table

Note 01 mentioned that the API's OpenAPI document generates an MCP toolset. The interception layer joined it: arming a rule, scheduling or cancelling a callback, pulling a run's recorded traffic are tools an agent calls by name, around forty in all, under the same run-token accounting and the same keys. The browser got the same operations as a page of one-click fixtures, and a dashboard that reads the test results CI pushes to it, flags flaky tests and shows how far each deployment lane is behind. Humans, CI and agents drive one surface, which is the only reason the three stay in agreement.

The details that bit

  • Content-Length is always set on a mock. One provider's client re-sends a request whose reply arrived chunked. In a top-up flow that is a duplicate top-up. Found the hard way.
  • Mock the status the real provider sends, not the one the sandbox sends. The sandbox that reported "submitted" forever had quietly taught a test to accept it.
  • Templates are not free. One provider's catalogue is 47 MB. Rendering it through a template engine on every request was the difference between a tool that idles at 300 MB and one the cluster evicts. Static bodies are now served as bytes, compiled templates are cached, and the common reply is 146 bytes.
  • Never run two of it. Rules and pending callbacks live in memory and are saved as whole sets, so two overlapping pods would miss each other's rules and send callbacks twice. One replica, recreate on deploy, written down where the next person will look.
  • Partial mocks need explicit catch-alls. While a provider is half mocked, its list ends with forward-everything rules. Otherwise the armed-but-no-rule 500 fires on the first call nobody thought of, which is correct and still ruins someone's afternoon.
  • A QA tool is used at the end of bad days. The home page has an arcade: Prod Invaders, Merge Conflict, Missile Command: On Call. It costs nothing and it is the page people remember.

What it solved

5 / 0top-up providers fully mocked on dev, calls forwarded to them
13providers behind one surface, one port each
~40MCP tools an agent can call, under run-token accounting
47 MB → 146 Bthe common catalogue reply, before and after serving bytes
  • Dev stopped paying for tests. The five providers that carried most of the traffic are answered locally. Sandbox outages and rate limits left the critical path.
  • The failure paths became tests. Declined, reversed, timed out, called back late: each is a rule a test arms for its run, and tears down with it.
  • Every answer has a provenance. Which rule, which run, which body. A red test on a mocked provider is debuggable from the recorded exchange, not from guesswork.
  • Three callers, still one surface. The suite, QA in a browser and AI agents arm the same rules and read the same traffic. Note 01's "three callers" problem did not come back.

The pattern underneath both notes is the same: find the state nobody owns, give it an owner and a token, and make every failure visible. Next in the series: deleting a monolith in waves. Questions or war stories of your own: .