MultiStoreSearch Toolkit — Multi-Origin Retrieval for Agents

MultiStoreSearchToolkit fans a query out across every retrieval plane AI-Parrot supports — vector stores, PageIndex, GraphIndex, and the ParrotWiki (LLM-Wiki) — and gives the agent both each origin's native ranking AND a single merged, deduped, BM25-reranked top-k list.

[!TIP] Origins are individually enable/disable-able — just construct the origins list with the adapters you want. An agent with only a VectorStoreOrigin behaves exactly like the old vector-only search; add ParrotWikiOrigin or GraphIndexOrigin to widen the plane.

---

Table of Contents

• Quick Start
• The Four Tools
• Origin Adapters
• The Response Payload
• FTS Capability Matrix
• Caveats & Known Limits
• Migration Note (FEAT-379)
• API Reference

---

Quick Start

import asyncio
from parrot_tools.multistoresearch import MultiStoreSearchToolkit
from parrot_tools.multistoresearch.origins import (
    VectorStoreOrigin,
    ParrotWikiOrigin,
)

async def main():
    toolkit = MultiStoreSearchToolkit(
        origins=[
            VectorStoreOrigin(store=my_pgvector_store, name="pgvector"),
            ParrotWikiOrigin(store=my_wiki_store),
        ],
        k=10,             # merged top-k size
        k_per_origin=20,  # candidates requested from EACH origin
    )

    # Agent-facing tools (auto-generated by AbstractToolkit):
    tools = toolkit.get_tools()  # store_search, batch_search, fts_search, list_search_origins

    # Or call directly:
    response = await toolkit.store_search("what is an endcap?")
    for section in response.sections:
        print(section.origin, section.status, len(section.hits))
    print("merged:", [hit.content[:40] for hit in response.merged_top_k])

asyncio.run(main())

MultiStoreSearchToolkit also satisfies the core MultiSearch protocol (parrot.models.MultiSearch) via its search() method — pass a toolkit instance directly to AbstractBot.configure_store_router(multi_store_tool=toolkit) for StoreRouter's FAN_OUT fallback policy.

---

The Four Tools

Tool	Purpose
store_search(query, k=None)	Search all enabled origins; returns grouped sections + merged top-k.
batch_search(queries, k=None)	N queries × M origins in ONE asyncio.gather — efficient for batch RAG jobs.
fts_search(query, k=None)	Runs only on FTS-capable origins; non-capable origins appear as "skipped" sections.
list_search_origins()	Static configuration view: name, kind, description, FTS capability, timeout, adapter-specific settings (e.g. PageIndex mode). No live health probing.

Each method's docstring IS its LLM-facing tool description — the agent sees exactly what's documented above.

---

Origin Adapters

Every adapter implements the SearchOrigin contract (parrot_tools.multistoresearch.origins.base.SearchOrigin): name, kind, description, supports_fts, timeout, async search(query, k), and optionally async fts_search(query, k).

VectorStoreOrigin — pgvector / FAISS / ArangoDB

Wraps any duck-typed store exposing async similarity_search(query, limit=...). FTS leg auto-detected via a callable fulltext_search attribute (ArangoDB).

from parrot_tools.multistoresearch.origins import VectorStoreOrigin

pgvector_origin = VectorStoreOrigin(store=pgvector_store, name="pgvector")
arango_origin = VectorStoreOrigin(store=arango_store, name="arango")
print(arango_origin.supports_fts)  # True — ArangoDBStore.fulltext_search exists

PageIndexOrigin — vectorless, tree-based reasoning RAG

Exposes PageIndex's three retrieval backends behind one mode switch. Default is hybrid (balanced cost/quality). Pass exactly the backend matching your chosen mode:

from parrot_tools.multistoresearch.origins import PageIndexOrigin

# hybrid (default) — BM25 + optional LLM-walk + optional dense signals
hybrid_origin = PageIndexOrigin(hybrid=hybrid_page_index_search)

# llm — LLM reasons over the tree directly. SPENDS TOKENS PER CALL.
llm_origin = PageIndexOrigin(llm=page_index_retriever, mode="llm")

# vector — brute-force cosine similarity; requires an async embed_fn
async def embed_query(text: str):
    return await my_embedder.embed(text)

vector_origin = PageIndexOrigin(
    vector=flat_matrix_search, embed_fn=embed_query, mode="vector",
)

mode is validated at construction (ValueError for anything outside {"vector", "hybrid", "llm"}, or when the matching backend/embed_fn is missing) — never at search time. The vector mode's underlying FlatMatrixSearch.search is synchronous; the adapter offloads it via loop.run_in_executor so it never blocks the event loop.

GraphIndexOrigin — 4-phase graph-expanded retrieval

Wraps GraphExpandedRetriever (seed → expand → community annotation → assembly). Optionally configured with a SQLiteGraphReader to enable a full-text symbol search leg:

from parrot_tools.multistoresearch.origins import GraphIndexOrigin

# Search-only (no FTS leg)
graph_origin = GraphIndexOrigin(retriever=graph_expanded_retriever)

# With FTS leg (supports_fts becomes True)
graph_origin_fts = GraphIndexOrigin(
    retriever=graph_expanded_retriever,
    reader=sqlite_graph_reader,
)

fts_search delegates to reader.search_symbols(query, limit=k). Its FTS5/BM25 scores are negative, ascending = best match — the adapter preserves the reader's rank order and carries the raw score through unchanged (see OriginHit.metadata["score_convention"]).

ParrotWikiOrigin — the ParrotWiki / LLM-Wiki plane

Calls BaseWikiStore directly (search_fts / search_vector) — does NOT delegate to WikiCombinedSearch. Always FTS-capable; the vector leg only activates when an async embedder is configured (search_vector needs an embedding, not free text):

from parrot_tools.multistoresearch.origins import ParrotWikiOrigin

# FTS-only (no embedder — lexical search)
wiki_origin = ParrotWikiOrigin(store=wiki_store)

# FTS + dense vector leg
async def embed_query(text: str) -> list[float]:
    return await my_embedder.embed(text)

wiki_origin_vec = ParrotWikiOrigin(store=wiki_store, embedder=embed_query)

# Restrict to one category
docs_origin = ParrotWikiOrigin(store=wiki_store, category="docs")

---

The Response Payload

store_search, batch_search, and fts_search all return (or return a list of) parrot.models.MultiSearchResponse:

class MultiSearchResponse(BaseModel):
    query: str
    sections: list[OriginSection]   # one per configured origin, native order
    merged_top_k: list[OriginHit]   # BM25-reranked + deduped across ALL origins
    notes: list[str]                # e.g. the score-comparability caveat

Each OriginSection keeps that origin's native ranking intact — including duplicates across origins, which is intentional: the LLM should see each origin's genuine view of the world.

class OriginSection(BaseModel):
    origin: str
    origin_kind: SearchOriginKind
    description: str          # LLM-readable explanation of this origin
    status: str                # "ok" | "error" | "timeout" | "skipped"
    note: Optional[str]        # error/timeout/skip explanation
    hits: list[OriginHit]       # empty on error/timeout/skipped

merged_top_k is built by BM25-reranking ALL hits across all origins, then deduping (exact ID match first, then content hash) — the highest BM25-ranked occurrence of a duplicate wins. It is capped at k.

class OriginHit(BaseModel):
    id: Optional[str]
    content: str
    score: Optional[float]     # origin-native; NOT cross-origin comparable
    metadata: dict[str, Any]
    origin: str                 # adapter name, e.g. "pgvector", "wiki"
    origin_kind: SearchOriginKind
    native_rank: int             # 1-based position in the origin's OWN ranking

---

FTS Capability Matrix

Origin	supports_fts	Notes
ParrotWiki (ParrotWikiOrigin)	✓ always	search_fts is native to every BaseWikiStore backend
ArangoDB (VectorStoreOrigin)	✓	only when the wrapped store exposes callable fulltext_search
GraphIndex (GraphIndexOrigin)	✓ with reader	requires a SQLiteGraphReader; False without one
PgVector (VectorStoreOrigin)	✗	no Postgres FTS support (out of scope by design)
FAISS (VectorStoreOrigin)	✗	vectors only — FTS is impossible
PageIndex (PageIndexOrigin)	✗ always	no FTS leg in any mode

fts_search reports non-capable enabled origins as "skipped" sections with a reason, rather than silently omitting them.

---

Caveats & Known Limits

• Origin scores are NOT cross-origin comparable. A vector distance, a wiki BM25 rank, and a graph combined_score live on different scales. merged_top_k's ORDER is BM25-over-content only — origin-native OriginHit.score values are never blended into that ranking.
• Per-origin timeout defaults to 30 s, overridable per adapter via its timeout constructor argument. A timed-out or failing origin degrades to a section status note ("timeout" / "error") — it never fails the whole call.
• PageIndex llm mode spends LLM tokens on every call and can run close to (or past) the timeout — prefer the default hybrid mode unless you specifically need LLM tree-walk reasoning.
• No Postgres FTS. fts_search only covers origins that are FTS-capable today (wiki, Arango, GraphIndex-with-reader); PgVectorStore has no full-text method.
• list_search_origins() is static-only (v1) — it reports configuration, not live health or staleness.

---

Migration Note (FEAT-379)

The old single-tool MultiStoreSearchTool (multi_store_search registry entry) has been removed — clean break, no deprecation shim.

Before	After
parrot_tools.multistoresearch.MultiStoreSearchTool	removed — import raises ImportError
Registry key "multi_store_search"	Registry key "multi_store_search_toolkit" → MultiStoreSearchToolkit
Vector-only search (pgvector/FAISS/Arango)	Four origin families: vector stores, PageIndex, GraphIndex, ParrotWiki
Single flat top-k list	Grouped-by-origin sections + merged BM25 top-k
StoreRouter FAN_OUT called MultiStoreSearchTool._execute()	StoreRouter FAN_OUT calls any parrot.models.MultiSearch-satisfying object's search() — MultiStoreSearchToolkit satisfies it directly

If you constructed the old tool with pgvector_store=..., faiss_store=..., arango_store=..., migrate to:

toolkit = MultiStoreSearchToolkit(origins=[
    VectorStoreOrigin(store=pgvector_store, name="pgvector"),
    VectorStoreOrigin(store=faiss_store, name="faiss"),
    VectorStoreOrigin(store=arango_store, name="arango"),
])

---

API Reference

• parrot_tools.multistoresearch.MultiStoreSearchToolkit
• parrot_tools.multistoresearch.origins.SearchOrigin (contract)
• parrot_tools.multistoresearch.origins.VectorStoreOrigin
• parrot_tools.multistoresearch.origins.PageIndexOrigin
• parrot_tools.multistoresearch.origins.GraphIndexOrigin
• parrot_tools.multistoresearch.origins.ParrotWikiOrigin
• parrot.models.SearchOriginKind, OriginHit, OriginSection, MultiSearchResponse, MultiSearch

See also: PageIndex, LLM Wiki.