Unlock content inside of images easily with this new search capability in Office 365.
Earlier this year, we rolled out automatic detection of images that are uploaded to SharePoint and OneDrive. This intelligence identifies whether an image is a whiteboard, a receipt, outdoors, a business card, an X-ray and many other types. You can then search for ‘whiteboard’ and you’ll see all the whiteboard photos you’ve captured and uploaded.
Now, as we announced at Ignite, any printed words in an image are automatically detected, extracted and made searchable. Using computer vision technology, when you upload the image, the location data (if available) from a photograph (such as Oslo, Norway), and the identification and extraction of text will happen automatically and become searchable. You can search in SharePoint, OneDrive or Office.com to find your captures.
Use visual content intelligence to simplify your work life
Many people complete expense reports for travel. While at a restaurant, snap a photo of the receipt. You can do this directly from the OneDrive mobile app, Office Lens mobile app, or just upload a photo you’ve taken with your device. Later on, when you go to file your expenses, you don’t have to remember where you stored it, but instead can search for something that you remember about the expense, for example ‘sushi’ or a location.
We’re excited to bring you this new capability and would love to hear how you use it and what ideas you have to make the service better. Let us know in the comments, or submit new ideas to onedrive.uservoice.com.
Frequently asked questions
What kinds of images can be made searchable?
We create previews and thumbnails for the types listed below:
We only tag and run OCR for items containing text on files of type: "jpg", "jpeg". "png".
There’s a great range with 21 different file formats including common ones such as "bmp", "png", "jpeg", "jpg", "gif", "tif", "tiff", "raw", and also "arw", "cr2", "crw", "erf", "mef", "mrw", "nef", "nrw", "orf", "pef", "rw2", "rw1", "sr2".
What languages are supported?
Text extracted from an image is in the language captured from the image and is searchable in that language.
The detection of the image type right now is only English. For example, a receipt, business card whiteboard. In the future, we’ll automatically look at the language set on the SharePoint site that the image was uploaded to and translate the type into that language. In the case of OneDrive, we’ll translate it to the language you have set in your preferences.
What other features do you have planned?
We really want to connect your captures to workflows. The goal is to look at what the object is and take action based on it, via Flow or PowerApps, so we can help you move your work forward. We also will learn from patterns you have with types of objects – personalized learning, as part of the Microsoft Graph, to suggest actions and perform them automatically for you after the pattern is established.