You Shipped an MCP Server or API.
Can Agents Actually Use It?

Agents pick tools by matching your description against the user's task. In our testing, that's the single highest-leverage thing you can change. Brand and name recognition mattered surprisingly little when we tested it directly: anonymizing server names barely moved selection. The words themselves are what matters.

So — describe the job, not the endpoint. "Get task data" tells the model nothing. "Search tasks by project, assignee, status, or due date" tells it exactly when to reach for you.

Another potential pitfall is easy to overlook: the model's own training data. When an agent gets a question, it can call your endpoint — or answer from what it already knows, instantly and for free. Give it a reason not to! If your data is live, private, authoritative, or the product of computation the model can't do from memory, say that in the description. Give the agent the novel-data catnip it so deeply craves.

In the OpenAI tool interface we tested (August 2026), requests exposing more than 128 tools were rejected before inference. Not degraded — rejected, so the model never sees any of them. We verified it at runtime: 130 tools returns an error; 128 goes through. Treat it as a constraint of eager tool loading, not a universal law — clients that defer tool loading can expose bigger surfaces.

This cap applies to the client's entire assembled toolset, not just yours. If a user loads your 40 tools next to two other servers, you're sharing that 128 budget — and someone gets cut. Assume you get a slice, not the whole thing.

Big surfaces also tax you on every call: in clients that load your full surface into context eagerly, the model reads every definition each time, even when it uses one tool. Deferred tool loading reduces that tax (where the client supports it). And most big surfaces carry tools nobody exercises; on one 34-tool API we tested, 16 tools were never called by any model across three full test runs of our task suite.

(One safety rule bundling should respect: bundle reads freely; keep writes — especially destructive ones — as their own tools, because clients grant permissions per tool name.)

The trendy version of the use-small-tool-counts argument says big tool lists confuse models into misusing them. Surprisingly, that's not actually what we've measured.

Below the caps, call quality stayed near-ceiling — 99.5% schema-valid and 98.8% semantically sensible across 5,300+ calls on developer-tool surfaces up to 100 tools. The measured costs of a big surface are the cap, the token tax, and the unexercised surface — not confusion. Fewer, more flexible tools is still the pattern, though in our data the reason is economic, not model collapse.

A developer reads your docs, finds the ID format, and moves on. An agent gets "what's ethereum trading at?" and has to bridge from human words to your input format on the fly.

In our corpus, APIs that accept natural inputs — city names, ticker symbols, owner/repo strings — sat near zero friction. APIs that demand internal IDs, slugs, or coordinates forced extra discovery calls before any useful work. That's cross-API correlation, not a same-API A/B test — but every extra call is another chance to fail, another hit to your rate limit, and more cost for whoever runs the agent.

Three refinements keep this safe. Keep accepting canonical IDs alongside the human words. When a name matches several things, return the candidates. And for destructive operations, require the canonical ID: "delete the project called Website Redesign" should never fuzzy-match its way to the wrong one of three.

This is arguably the most straightforward high-leverage fix on the list. On one API we audited, the most common hard failure had exactly this shape: the model guessed an asset ID, got a bare 404 back, and had no route to an answer it was one call away from.

The fix is one sentence. Put "use list_assets to find valid IDs" in the lookup tool's description. Then put it in the 404 body too — "not found, try /assets?search=" gives a stuck agent its next move. A bare "not found" gives it nothing, and nothing is how retry loops start.

(On a brief don't-get-prompt-injected note, be aware that models may treat hint text as instructions, so treat every hint as an instruction-bearing surface. Keep imperative hints entirely server-authored and templated. Treat echoed user content as untrusted data; keep it structurally separate from the guidance, never part of it. An error body that reflects attacker-controlled text into imperative guidance is a recipe for a bad day.)

The general rule: every failure state should tell the agent the cause, whether retrying can help, and the next action. That applies well beyond 404s. An auth error that names the missing scope is recoverable; a bare 403 is a dead end the agent will retry against anyway, burning its budget on the wrong interpretation.

Don't assume parameter count is the problem. Models handled 8-parameter tools fine when the names explained themselves — and ignored 2-parameter tools with cryptic names. (Eight clear scalars is not a license for 25 nested, coupled fields — but fix the names before you delete capability.)

And prefer the conventional name where one exists — limit, fields, query. A merely descriptive invention is a downgrade from a name every client and model already handles.

In our corpus, agents averaged ~2.5 API calls per completed task — and 4.4 on the heaviest vertical: discovery, the real call, a retry, maybe a comparison. Free tiers sized for a human clicking one thing at a time break the moment agents show up.

We watched this play out by task type: simple lookups completed even through throttling, while multi-call tasks — comparisons, screening — collapsed, because the fourth call in the sequence is the one that hits the limit. If agents matter to you, your rate limit is a product decision now, not an infrastructure default.

When your API returns partial data, the model doesn't bat an eyelash; it often fills the gaps with invented values that look exactly like real ones: realistic prices, plausible dates, and confident precision. We've watched this happen across model families.

Clearly you don't want to make models hallucinate like they're at a Grateful Dead show in 1972. But how can you avoid this?

It turns out the don't-make-shit-up checklist is fairly simple. Don't return an empty result wrapped in a 200 that reads as success. Don't return an error message with a success status code. And when data is partial, say so explicitly; a completeness flag lets the model distinguish "that's all there is" from "data may be missing," instead of improvising the rest.

The broader principle, where it fits your data: expose freshness, units, provenance, and whether a field is genuinely null versus simply unavailable.

Here's another failure that will likely be missing from your logs: the agent calls your endpoint, your endpoint dutifully returns everything… and proceeds to flood the context window. Two steps later the task dies mid-answer, looking nothing like a server problem when in fact it's something you, dear endpoint publisher, have total control over.

We measured this as its own failure stage. On the search-shaped services in our corpus, useful data often arrived and the answer still didn't: payloads were useful 93–99% of the time, while tasks completed only 42–67% of the time. The data arrived. The answer died anyway.

The fix directions: size your default pages in tokens, not rows. Offer a fields parameter so an agent can pull three fields instead of forty. Return every slice with its metadata — returned, total, a cursor — so the model knows it's holding a slice. And when you truncate, say so explicitly; silent trimming turns into confident wrong answers downstream.

Every service we've tested fails in its own place. Our weather vertical's failures were execution failures — timeouts, concentrated in one unreliable server while its neighbors ran clean. Four search services executed perfectly and failed on payload quality. Crypto testing was dominated by rate limits and access tiers. And one large devtool surface failed before a single call — too many tools to load. A better description fixes none of those. Know your stage first.

Then break results down by task type, because averages lie. And watch how hard agents work, not just whether they succeed: some models brute-force through a broken flow with retries. The task completes, the score looks fine, and the fragility is invisible until a cheaper model or a stricter token budget shows up. High effort is your API's friction being paid for by the incoming agents' inference bills.