# Connect your agent to your API (adapt to your setup)
Source: https://speakeasy.com/blog/cost-aware-pass-rate

AI benchmarks have a giant blind spot: they celebrate agents that succeed at any cost. Admiring an agent's position on today's leaderboards is like praising a brute force search algorithm for its 100% success rate. If accuracy is the only metric, brute force is perfectly acceptable.

Bigger AI models win by throwing more parameters and reasoning cycles at problems, because this is what the benchmarks reward. Even a stopped clock is right twice a day, but that doesn't make it a good timepiece.

New research from Bloomberg shows that by taking cost into account while optimizing tool descriptions and agent instructions, we can achieve similar or even better performance at a fraction of the cost.

The paper, [A Joint Optimization Framework for Enhancing Efficiency of Tool Utilization in LLM Agents](https://web.archive.org/web/20250818044410/https://aclanthology.org/2025.findings-acl.1149/), introduces a new measure for how well language models use tools: Cost-Aware Pass Rate (CAPR). Instead of merely asking, "Did the model use the tool?" the study adds, "and at what cost?"

Anyone building agents or tools for agents could benefit from this research. We'll summarize the key finding here and look at a practical example of how to implement the joint optimization framework for your own APIs and agents.

## The research: Context vs inference scaling

The Bloomberg team tested their joint optimization framework across 16,464 APIs in StableToolBench, and against real-world scenarios in RestBench. They compared two fundamental approaches: **Context optimization** (improving instructions and tool descriptions) versus **inference scaling** (adding more reasoning steps, like Chain-of-Thought[^1] or tree search[^2]).

[^1]: **Chain-of-Thought (CoT)** encourages models to generate intermediate reasoning steps before arriving at a final answer. Rather than jumping directly to conclusions, CoT prompts guide models to "think out loud" through problems step by step, significantly improving performance on complex arithmetic, commonsense, and symbolic reasoning tasks.

[^2]: **Tree search methods** like depth-first search (DFS) extend linear reasoning approaches by exploring multiple solution paths simultaneously. When applied to LLM tool use, these methods maintain a backup mechanism to quickly restart from the last successful state when a tool call fails, theoretically improving both effectiveness and efficiency compared to simple retry strategies.

The research suggests that context optimization leads to better performance and lower costs compared to inference scaling alone.

When agents only have access to vague tool descriptions, they often resort to trial-and-error approaches, stubbornly repeating slightly varied queries until they stumble on a successful one. Sophisticated reasoning algorithms can't compensate for poor documentation.

## Is this just prompt engineering?

Not quite. While improving prompts is an important part of the solution, tool descriptions may play a bigger role.

Perhaps the most surprising finding from this research is that only improving tool descriptions results in a bigger gain than only doing instruction tuning. While **joint optimization** (improving both tool descriptions and agent instructions) leads to the best outcomes, good tool descriptions can actually be more important than good prompts.

## Why context quality matters more than we thought

LLMs select tools and make tool calls based entirely on the available context. Tools with incomplete or vague descriptions cause agents to iterate through multiple attempts before finding and using the right tool effectively.

This is compounded by incomplete instructions or guidance on how to use the tools.

When tools depend on each other, failures can cascade, leading to a situation where the agent's inability to effectively use one tool impacts its interactions with others. Improving a single tool's description can lead to better performance across a suite of interdependent tools.

Giving an agent more reasoning capabilities, like the ability to perform Chain-of-Thought reasoning about which tools to pick and how to use them, may reduce the impact of these issues. However, inference scaling doesn't make up for missing context.

The research data shows that context optimization provides 10-30% cost reduction at fixed accuracy, while inference scaling increases costs by 200-500% for marginal accuracy gains.

## Practical implementation: The joint optimization framework

Here's how to implement the Bloomberg research team's joint optimization approach for your own APIs and agents. The process works with any tool-calling system, whether it's MCP servers, function calling, or custom APIs.

### The challenge: Generic tool descriptions

Most API documentation follows patterns like these:

- `listItems`: "List all items"
- `createItem`: "Create a new item"
- `updateItem`: "Update an item"
- `deleteItem`: "Delete an item"

While technically accurate, these descriptions don't help LLMs choose between similar operations, understand required parameters, or handle edge cases effectively.

### Step 1: Establish baseline measurement

First, connect your agent to your existing API and collect interaction data. The key is measuring both success and efficiency:

```python

agent = YourAgent(
    system_prompt="Your current system prompt",
    tools=your_existing_tools
)