What is entity extraction?
Entity extraction identifies the named things in a piece of text and labels what kind of thing each one is. In retrieval it is applied in two places. Applied to documents during ingestion, it records which entities each passage mentions. Applied to a question at query time, it identifies the specific names the person is asking about, so the system can look for those names precisely instead of relying on loose similarity.
Also called: Named entity recognition (NER), Query entity extraction
Why a question about a named thing needs an exact match
Meaning-based retrieval treats a policy number or a product code as just more words, and it will happily return a passage about a similar product instead of the one named. For enterprise questions that is usually the wrong answer delivered with confidence. Extracting the entities from a question lets the system filter by them and start graph lookups from them. The general similarity search then works inside that narrower and more accurate set.
- Identifiers and codes are where similarity search is weakest.
- Entities in a question can be used as hard filters on retrieval.
- They provide starting points for traversal in a graph index.
- Extracted entities make answers easier to verify against their sources.
How entity extraction works
- A model reads the question and labels each named entity with its type.
- Each entity is normalised against the forms used in the organisation's records.
- Recognised entities are matched to nodes in the graph index where one exists.
- Retrieval is filtered or weighted toward passages that mention those entities.
- Unrecognised entities are passed through as keywords for exact matching.
- The entities used are recorded with the answer for later review.
How entity extraction is evaluated
Entity extraction is evaluated on labelled questions and passages drawn from real usage. Measure the following on that sample:
- precision and recall of extracted entities against the labels
- share of entities correctly normalised to the organisation's canonical form
- retrieval accuracy on entity questions with and without extraction
- rate of entities assigned the wrong type
- latency added at query time
Limitations, risks and common failure modes
- Internal abbreviations are missed because the model has never seen them.
- Similar product names are merged into a single entity.
- A common word that is also a product name is extracted everywhere.
- Strict filtering on a misread entity removes the passage that held the answer.