Home

Search for words in your images in Office 365

Unlock content inside of images easily with this new search capability in Office 365.

Earlier this year, we rolled out automatic detection of images that are uploaded to SharePoint and OneDrive. This intelligence identifies whether an image is a whiteboard, a receipt, outdoors, a business card, an X-ray and many other types. You can then search for ‘whiteboard’ and you’ll see all the whiteboard photos you’ve captured and uploaded.

 

Now, as we announced at Ignite, any printed words in an image are automatically detected, extracted and made searchable. Using computer vision technology, when you upload the image, the location data (if available) from a photograph (such as Oslo, Norway), and the identification and extraction of text will happen automatically and become searchable. You can search in SharePoint, OneDrive or Office.com to find your captures.

 

Use visual content intelligence to simplify your work life

 

Many people complete expense reports for travel. While at a restaurant, snap a photo of the receipt. You can do this directly from the OneDrive mobile app, Office Lens mobile app, or just upload a photo you’ve taken with your device. Later on, when you go to file your expenses, you don’t have to remember where you stored it, but instead can search for something that you remember about the expense, for example ‘sushi’ or a location.

 Search_Image_Coffee.png

 

 

We’re excited to bring you this new capability and would love to hear how you use it and what ideas you have to make the service better. Let us know in the comments, or submit new ideas to onedrive.uservoice.com.

 

Frequently asked questions

 

What kinds of images can be made searchable?

 

There’s a great range with 21 different file formats including common ones such as "bmp", "png", "jpeg", "jpg", "gif", "tif", "tiff", "raw", and also "arw", "cr2", "crw", "erf", "mef", "mrw", "nef", "nrw", "orf", "pef", "rw2", "rw1", "sr2".

 

What languages are supported?

 

Text extracted from an image is in the language captured from the image and is searchable in that language.

 

The detection of the image type right now is only English. For example, a receipt, business card whiteboard. In the future, we’ll automatically look at the language set on the SharePoint site that the image was uploaded to and translate the type into that language. In the case of OneDrive, we’ll translate it to the language you have set in your preferences.

  

What other features do you have planned?

 

We really want to connect your captures to workflows. The goal is to look at what the object is and take action based on it, via Flow or PowerApps, so we can help you move your work forward. We also will learn from patterns you have with types of objects – personalized learning, as part of the Microsoft Graph, to suggest actions and perform them automatically for you after the pattern is established.

 

Happy finding!

 

Naomi

25 Comments

Sounds good! So will it become possible to upload images automatically from the OneDrive mobile app to OneDrive for Business? Will the Windows Photos app support images stored in OneDrive for Business? Aso. Aso.

 

And yes, I did vote for it on UserVoice, feedback hub and so on :)

New Contributor

"we’ll automatically look at the language set on the SharePoint site"
That may not be a useful indicator of the desired language of the image type.  The current language of the user is closer, like in OneDrive.  Why do the types need to be translated, why not let the MUI do it?

Having the text itself extracted using language detection is good, but your receipts might be in a variety of languages, so you'll have to remember to also search for kaffe or kahvi.

Frequent Visitor

Any support for on-premise installations?

Occasional Contributor

Not sure what the roll out situation is from this article. Is this an expansion on the announcement form Ignite, in Targeted Release or at General Availability?

Microsoft

Benjamin Haynes - the rollout is stated in the blogs.office.com article - see below:

Text in image search is currently rolling out to Office 365 commercial subscribers and will be available worldwide by the end of 2017.

In the past, if a PDF was uploaded as an image (scanned document), SharePoint did not OCR the PDF document and the text was not searchable. With this implementation, will PDF's be searchable, if scanned as an image?

Senior Member

You mention 21 different file formats but unfortunately not PDF, is this going to be implemented as well or are PDFs excluded from this search feature at all?

 

Many of our users do not understand the difference between searchable OCR PDFs and non OCR PDFs, therefore they are mostly disappointed if search does not show all expected files.

 

In a global organization where you have different scanners wit and without OCR it is a nightmare to explain people that it is about the document and not about the search why they won’t find non OCR PDFs in the search results.

New Contributor

I really don't care to much about pictures but PDF's are huge, we now have to go through and fix all the pdf's that have been uploaded that were not OCR before uploading .

 

New Contributor

Hello,

 

I am not able to use this functionality. Is there any pre-configuration required to use this functionality. I have uploaded my business card in asset library in SharePoint and now not able to search using any word of my business card. I have also uploaded business card in OneDrive and same thing happen in onedrive also. 

New Contributor

Has there been any response on this with regards to non-OCR PDFs, images of scanned documents within a PDF, being processed and becoming text-searchable?

New Contributor

It looks like this would be very helpful to me.

But I could not figure out from your article, how I activate the capability.  is it a now APP?  An addition to an existing Office program?

 

Please advise.

Occasional Visitor

This photo intelligence feature don't work with my Onedrive for Business account. Is this already implemented ? I made a photo in JPEG with a really clear text inside and uploaded it to my OneDrive map. But afterwards the search function in OneDrive couldn't find my photo... ?

Frequent Visitor

Where is the data analyzed and stored for OCR? Is it in the tenant?

Senior Member

Any Microsoft representatives here?

 

Unfortunately there are more questions than answers in this blog right now.

 

Is this search feature also available für PDF files or at least is it on Microsoft's development roadmap?

How can we find out if this Feature is already rolled out on our tenant?

 

Best Regards,

Mehmet

Regular Visitor

Agree with Mehmet.

The article is confusing rather than informing. It is half of February and feature is not working.

Would be good if anyone would have answered here...

Trusted Contributor

@Naomi Moneypenny could you get someone to answer these questions

@Michael Holste

New Contributor

It would be useful to know how this will work with an organization's compliance rules. For example, if a user uploads a bunch of photos that turn out to have PII (Social Security numbers, credit card numbers, etc.) in them, will that get flagged in some way? Will the user be notified? Will an admin be notified?

Senior Member

@Naomi Moneypenny

@anyone from Microsoft who takes care for customers

 

What is the intention of your post? Why has this been added to this panel?

 

You announced a new feature in a tech community panel but your are neither here to discuss with us nor answering any of our questions - this is very frustrating.

 

Happy to get an answer at least on this question.  

 

Regards Mehmet

Hello everyone, I've been trying to gather some answers to your questions. This feature has been completed rolled out at the end of last year. PDF files are generated by many different applications which has consequences for how those documents are made searchable. Even though as an end user, it appears that a PDF is one format, how the PDF is created makes a big difference in how to make it searchable. In SharePoint there is already a search function makes many types of PDFs searchable. There's no plans currently to extend the work of the image recognition team to PDFs imminently but engineering is aware that this is a concern, but there are many nuances to how to make this cover every situation. The data extracted is processed and lives wherever the data is stored, which includes geo support for data sovereignty. Hope this helps.
New Contributor

The previous Microsoft announcement stated that there was no a way in which we could take an image that contained text, and produce the text in digital form.

 

Is this a capability that exists today?

 

If so, where and how do we find out how to utilize it?

 

Thank you.

Trusted Contributor
New Contributor

Thanks much.  This is a great capability.  Publicize it more.

Senior Member

Is the Intelligent search for images in OneDrive also available for OneDrive for Business, if not do you know when it will be?  Is this feature on the roadmap for business users?

 

Thank you

Frequent Contributor

This feature is still not working for us. Is it working for others?

New Contributor

If what you want to do is take a graphic file (a photo for example) where there is text on a sign or a screen shot of text or any graphic that contains text and extract the text from the photo, this works very well and we use it all the time.

 

The instructions are a little vague however.

 

You MUST use OneNote for this function, however.  At the moment it does not work in any other Office Program.

 

Once you realize that you have to use OneNote, the rest is easy.

 

  1. Copy the graphic from anywhere -- any application -- with Control C.
  2. Go to a page in OneNote
  3. Paste the graphic into that page using Control V
  4. Right Click on the Graphic You have just pasted
  5. Click on the Command "Copy Text from Picture"
    (This extracts the text from the photo and places it in the Clipboard)
  6. Go to any application you wish and Control V will paste the text from the picture (now in the clipboard) into the application where you want it.