Keyword Search Tied an 8B Embedding Model at Picking Agent Tools

What changed

A keyword search method matched the performance of an 8-billion-parameter embedding model (BGE-large) in selecting agent tools across 200 questions and nearly 3,800 tools. Running on a CPU, BM25—a classic keyword search algorithm—operated about 40 times faster than BGE-large. The top-performing setup used full tool documentation rather than just short descriptions or metadata, boosting accuracy.

Why builders should care

Choosing tools for agent workflows often involves embedding models that require significant computation and cloud resources. This experiment shows that traditional keyword search can deliver similar tool-matching quality while cutting costs dramatically by running efficiently on CPU hardware. It questions the automatic preference for large embeddings on GPU and hints that simpler methods can scale more cheaply and quickly for practical tasks like agent tool selection.

The practical takeaway

For anyone building or operating agent ecosystems, the takeaway is clear: do not overlook tried-and-true keyword search for matching intents to tool capabilities. Using full tool documentation in queries improves keyword search quality enough to rival expensive embeddings. This opens a cheap path to faster responsiveness without sacrificing accuracy when selecting appropriate tools or plugins from large catalogs.

What to watch next

Watch how vendors respond—will embedding providers integrate keyword search as a cost-saving fallback or hybrid? Also, monitor whether tool catalogs start optimizing documentation formatting to leverage keyword search accuracy improvements. Finally, observe speed and pricing shifts in agent tool marketplaces as this approach pressures purely embedding-based retrieval.

AI Quick Briefs Editorial Desk