Forum Discussion
Does Microsoft 365 Copilot Agent Builder support OCR and image analysis for SharePoint?
Hello !
The Microsoft 365 Semantic Index only indexes supported Microsoft 365 content types such as Word, PowerPoint, PDF, OneNote, and SharePoint pages. JPG and PNG files are not indexed in the same way. As a result, an agent cannot search or retrieve information contained solely within an image stored in a SharePoint library.
Could Copilot Studio overcome this limitation? Not really.
Copilot Studio can analyze an image when a user uploads it directly within a conversation. However, when the image is stored in a SharePoint library and used as a knowledge source, the limitation remains the same : the agent can only work with content that has been made searchable.
Microsoft recommends several approaches:
- SharePoint OCR (native OCR) Enable OCR on the library using SharePoint Premium (pay-as-you-go). The text extracted from images is indexed by SharePoint Search and becomes available to agents. This is the most straightforward option and does not require custom development
Yu can read more here : https://learn.microsoft.com/fr-fr/microsoft-365/documentprocessing/ocr-overview?view=o365-worldwide&source=docs
- AI Builder + Power Automate A Power Automate flow can be triggered when a new file is added to a library. AI Builder extracts the text from the image and stores it in SharePoint metadata. It is less suitable for very large volumes of documents...
- Power Automate + Azure AI Document Intelligence Similar to the AI Builder approach, but better suited to large document volumes and more complex document structures but this option requires an Azure service.
If the information exists only inside an image, that image must first be converted into searchable text through an OCR process, whether via SharePoint Premium OCR, AI Builder, Azure AI Document Intelligence, or another OCR solution. Only then can the agent access and use the information.
I hope this helps clarify things.