What is federated search?
Federated search queries several repositories where they are, instead of copying their content into a single index first. Each system answers for its own content under its own permissions, and the results are merged for the asker. Nothing is duplicated, so the question of a stale copy does not arise.
Also called: Distributed search
Why some repositories can never be copied into one index
The constraint that forces federation is usually written into a contract or a regulation rather than chosen by an architect. Clinical records and examination material cannot be duplicated into a vendor's index, whatever the retrieval benefit would be. Federation accepts a harder engineering problem in exchange for leaving the content where its owner can account for it. The trade is real, since ranking across systems that score differently is considerably worse than ranking within one index.
- Federation leaves content under the control of the system that owns it.
- Permissions are evaluated by each source, which removes a synchronisation duty.
- Freshness is inherent, because nothing is ever a copy.
- Ranking quality suffers, since scores from different systems are incomparable.
- Latency is bounded by the slowest repository in the set.
How federated search works
- Each repository exposes a query interface the federator can call.
- The question is translated into the query form each system understands.
- Calls proceed in parallel, with a timeout per repository agreed in advance.
- Returned results are normalised onto a common relevance scale.
- Merged candidates are reranked together before an answer is composed.
- Repositories that failed or timed out are declared in the response.
How federated search is evaluated
Federated search is assessed on what the merge does to quality and on what happens when a source is unavailable. Examine the following across the connected set:
- recall of the correct passage after merging against each source alone
- latency at the ninety-fifth percentile across the whole set
- frequency of a repository timing out or returning an error
- effect on answers when one source is excluded
- comparability of relevance scores between sources after normalisation
Limitations, risks and common failure modes
- Scores from different systems are merged as though they were on one scale.
- One slow repository determines the latency every user experiences.
- A silent timeout produces an answer that omits a whole source without saying so.
- Source systems lack the query expressiveness the federator assumes.
- Per-source rate limits make the design unworkable at production query volume.