What is image captioning?
Image captioning generates a written description of what an image shows. In training and knowledge systems it is applied to screenshots, diagrams, photographs and charts embedded in documents and slides. The caption makes the content of the image available to search and to a model answering questions, neither of which can interpret the image directly.
Also called: Image description, Alt text generation
Why procedures hidden in images are invisible to every other step
Standard operating procedures often carry their most important content in images, such as a screenshot of the correct field or a diagram of an approval flow. Text extraction ignores those images, so the material is never indexed and never taught. A learner who asks how to complete a step receives an answer drawn only from the surrounding text. Captioning brings the visual content into the same pipeline as everything else, provided each caption is checked against the image it describes.
- Screenshots and diagrams often carry the procedure the text only refers to.
- Captions make visual content retrievable in answers and searches.
- Learners using screen readers gain access to the same material.
- Captions give a model the context to explain a figure in conversation.
How image captioning works
- Images are detected and extracted from documents and slides during ingestion.
- Each image is passed to a vision model with its surrounding text as context.
- The model writes a factual description of the image content.
- Text visible inside the image is transcribed into the caption.
- The caption is stored alongside the image and indexed with the document.
- Captions on controlled material are reviewed before being published.
How image captioning is evaluated
Image captioning is evaluated by comparing captions with the images on a representative sample. Check the following points on each caption:
- share of captions that describe the image accurately
- accuracy of text transcribed from inside images
- captions containing details the image does not show
- retrieval of image-based answers with and without captions
- reviewer time needed per caption on controlled material
Limitations, risks and common failure modes
- Captions describe the layout of a screenshot and miss the instruction it shows.
- The model invents labels or values that are too small to read.
- Decorative images are captioned and clutter the index.
- Captions are not regenerated when a screenshot is updated.