Word search through thousands of pdf's?

Question

Is this the appropriate product to use if I want to create a word search index for thousands of pdf files, and then query it from my asp.net application?&nbsp;

luis cabrera-cordon · Answer

David_Bluebox&nbsp;
Azure Cognitive Search should work for this. If all your content is in SharePoint though I would first check if SharePoint search meets your needs.
&nbsp;
If you need additional flexibility that Azure Cognitive Search provides, you either could use the new SharePoint indexer (in preview:&nbsp;Configure a SharePoint Online indexer (preview) - Azure Cognitive Search | Microsoft Docs), or simply copy all your files to blob storage, and use the blob storage indexer.&nbsp; (Search over Azure Blob storage content - Azure Cognitive Search | Microsoft Docs)
&nbsp;
I hope this was helpful,
&nbsp;
Luis Cabrera, Azure Cognitive Search team.

luis cabrera-cordon · Answer

Yes, absolutely. You could put your thousands of pdf files in a repository like blob storage, then index it and then query for the information in those PDFs.

Note that there are many different types of PDFs. If you have scanned PDFs, you may want to add a skillset to your indexer that extracts text from the images embedded in the PDFs.

The easiest way to do all of this (in just a couple of minutes) is to follow this tutorial :

https://docs.microsoft.com/en-us/azure/search/cognitive-search-quickstart-blob

By the end of that quickstart, you will have an index that you can query so you can find any PDFs that meet the query requirements.

Thanks,
Luis Cabrera, Azure Cognitive Search Team

wavel · Answer

Last time I checked though, the system only indexed the first 30k or so of each document. I need the entire pdf (10k-2MB) searchable. These are all text pdf's by the way. No images.

david_bluebox · Answer

I have similar requirements where a client subscribes to a number of Market Research (in education) sources that provide regular PDF's containing text and images. My client wishes to perform a search across those documents e.g. all research documents related to students coming from China. The documentation I have read seems to also refer to the text limit as mentioned by Wavel. How can we execute such an indexing operation? All of the files currently reside in SharePoint.

luis cabrera-cordon · Answer

Wavel&nbsp;You should be able to index all of the content of the document.
&nbsp;
Note that different tiers have different limits in terms of the number of characters pushed into the index. See&nbsp;Service limits for tiers and skus - Azure Cognitive Search | Microsoft Docs.&nbsp;&nbsp;Note that that is not the limit for the size of the actual PDF.
&nbsp;
If you have an S2 tier for instance,8MB of characters get indexed, but it would take a pretty big PDF to generate 8MB of characters (about 2700 pages full of characters).

Forum Discussion

Word search through thousands of pdf's?

9 Replies

Resources