The 429 That Was Really a 400

A rate-limit error that was a malformed request, relabeled by the proxy's retry logic — and the self-sustaining loop that kept re-triggering it.

Summary

Our agent sessions were dying with what looked like rate limiting:

{'type': 'throttling_error', 'code': '429'}
No fallback model group found for original model_group=deepseek-v4-flash

It was not rate limiting. The proxy exhausts its retry budget against two identical hard 400s and relabels the result as a 429. The 400 comes from sending tool_choice with no tools — which every agent harness does, deterministically, on the last turn of a long session. And the failure sustained itself: the same 400 killed the context compressor, the un-compressed context caused the stream kills that burned the iteration budget, and hitting the iteration cap produced the exact request shape that 400s. One hook broke the loop at both ends.

The error that lies

Two things about the message are actively misleading.

"throttling_error / 429." The proxy is configured with two retries. Two identical 400s exhaust them, and what surfaces is a rate-limit classification. So you go and look at tokens-per-minute limits on your keys — we did, and there's even a well-known gotcha about new keys defaulting to a low limit, which makes the wrong trail feel warm.

"No fallback model group found." The obvious response is to configure a fallback group. The fallback receives the same malformed body and fails identically, so you've now built a second path to the same 400 and made the retry storm bigger.

When a proxy sits between you and the model, its error taxonomy is its own, not the server's. Get the upstream status code before you believe a classification. Ours took one curl straight at the inference server to settle.

The actual bug

The inference server rejects two shapes, both 400, both from the same validation function:

When using tool_choice, tools must be set.
tools must not be an empty array. Either provide at least one tool or omit the field entirely.

Measured on our gateway, with controls, because a bug like this is worth pinning exactly:

Why agent harnesses hit it every time: a harness with a tool-call iteration cap ends a long session with a terminal turn that says, in effect, you have reached the maximum number of tool iterations, no more tools are available. That turn correctly stops advertising tools — and leaves tool_choice: "auto" in the payload, because the field is set once when the request template is built.

So it isn't intermittent and it isn't load-related. Every session long enough to hit the iteration cap ends on a 400, and the busier the session the more certain it is. That's the profile that makes it look like throttling.

The config knob that should cover it doesn't. The proxy has a drop_params setting for exactly this class of problem. tool_choice is a fully supported parameter, so it passes through untouched — it is the combination of a valid parameter and an absent one that's invalid, and parameter-level filtering can't see combinations.

The loop

This is the part worth the post.

The agent's context-compression summarizer was itself sending tool_choice with no tools. It's a plain summarization call; it has no tools by definition. So it hit the identical 400:

Failed to get summary response: Error code: 429 ... When using tool_choice, tools must be set.

With the summary call dead, compression fell back to blindly dropping unprotected messages and logged:

Preflight compression made insufficient progress: ~293,521 -> ~283,935; skipping additional passes

Now trace it around:

1. The 400 kills compression.
2. Context stays at ~284k tokens.
3. Requests that large hit the 180-second no-data stream kill and return 502s.
4. Retries and failures burn the 25-iteration budget.
5. Hitting the iteration cap produces the terminal "no more tools" turn.
6. That turn sends tool_choice with no tools — the same 400.

Every symptom on that list had been investigated separately. The stream kills looked like a network or timeout problem. The context bloat looked like a compression-tuning problem, and we had spent days on compression settings that were, it turned out, already correct. The one fix broke the cycle at both ends, because both the compressor's call and the terminal turn are the same malformed shape. Compression went from 293k → 284k to 302,121 → 42,877 in a single pass.

When several unrelated-looking failures cluster in the same sessions, look for the one that both causes and is caused by the others. A self-sustaining loop presents as a list of independent bugs, and fixing any single symptom gives you a temporary, partial improvement that misleads you about the cause.

The fix, and where we chose to stop

A pre-call hook on the proxy that drops only provably meaningless fields: tools: [], and tool_choice of "auto" or "none" when no tools are present.

tool_choice: "required" and named-function choices still 400. That's deliberate. Those requests are asking for something the server cannot do, and silently reinterpreting them would produce a plausible answer to a question the client didn't ask. An unserviceable request should fail loudly rather than be quietly rewritten — the same reasoning that says a fallback group is the wrong fix here.

Three things that made the follow-up hard

A client update restarts daemons but not interactive sessions. The bug recurred four minutes after the client update landed, from a session that had been running the whole time and was still executing the pre-update module out of RAM. Restart the session before concluding a client-side fix didn't take.

Then it recurred from a session started 92 minutes after the update — which ruled that explanation out. At triage time the hook was loaded, its self-check passed on all four shapes, and live re-tests of both bad shapes returned 200. Loaded, wired, functional, and it had let one request through 40 minutes earlier. That points at the proxy's callback-capability cache, which memoizes on the number and identity of registered callbacks and can short-circuit the entire pre-call walk. An intermittent bypass of a correctly-installed hook is a caching bug, not a configuration bug — and the two look identical if you only check configuration.

A missing log line proved nothing. The hook logs at warning level, and warnings from that module never reach the container's log output. We spent time treating "no 'dropped' message" as evidence the hook hadn't run. Verify that your instrumentation is visible before using its absence as data — raise the level and confirm you can see it, then theorize.

The damage outlives the request

Six minutes after one of these 400s, a tool call started emitting malformed arguments — four times, rejected by the client's own local validator in 0.00 seconds:

tasks must be a JSON array of task objects; received a string that could not be parsed as JSON

That reads like an independent client bug, and it isn't. Compression died, the context stayed poisoned, and the model's output degraded accordingly. Check the error log for a preceding summarizer failure before blaming the client. Restarting the session "fixes" it only by discarding the poisoned context, which also discards the evidence.

A 429 that's really a 400 means the error is upstream of where you're looking — tracing that boundary is exactly what the setup audit does first.

Key takeaways

1. A proxy's error classification is not the server's. Two hard 400s exhausting a retry budget surfaced as throttling_error / 429. Get the upstream status before believing the label.
2. A fallback group is not a fix for a malformed request. The fallback gets the same body and fails the same way.
3. tool_choice with no tools is a 400, and every agent harness with an iteration cap sends exactly that on its terminal turn. It is deterministic, not load-related.
4. Parameter-level filtering can't catch invalid combinations. drop_params passes tool_choice through because the parameter is legal on its own.
5. Look for the failure that is both cause and effect. Ours killed compression, which caused stream kills, which burned the iteration budget, which produced the request that caused it.
6. Refuse unserviceable requests rather than rewriting them. We drop only provable no-ops; a required tool choice still fails loudly.
7. A client update doesn't restart running sessions, and a hook's absent log line is not evidence it didn't run. Confirm your instrumentation is visible before reasoning from silence.