Skip to main content
The Search backend gives the LLM two meta-tools: search_tools(query) to find relevant tools by description, and call_tool(tool_name, arguments) to invoke them. The LLM never sees your full tool list.

Setup

Requires sentence-transformers. Install separately: pip install sentence-transformers

How It Works

The key difference: instead of seeing all 100+ tools upfront, the LLM searches for what it needs. This is like a developer searching an API reference instead of reading the entire docs.

What the LLM Sees

Only two tools ever appear in the tool list:
Constant context cost regardless of how many tools you register:2 tool definitions instead of 200.

Configuration

When to Use

Use Search when you have a large API (100+ tools) where the LLM only needs a few tools per conversation.
Good fit:
  • Large APIs with 100+ tools
  • Tools with clear, descriptive names and docstrings
  • Exploration-heavy use cases (“what can this server do?”)
Bad fit:
  • Small APIs (Plain is simpler)
  • Strict ordering requirements (use stages)
  • Latency-sensitive apps (embedding adds ~50ms per search)