Forum Discussion
Tool for Azure cognitive search similar to Logstash?
My company has lots of data(Database: PostgreSQL) and now the requirement is to add search feature in that,we have been asked to use Azure cognitive search.
I want to know that how we can transform the data and send it to the Azure search engine.
There are few cases which we have to handle:
1. How will we transfer and upload on index of search engine for existing data?
2. What will be the easy way to update the data on search engine with new records in our Production Database?(For now we are using Java back end code for transforming the data and updating the index, but it is very time consuming.)
3.What will be the best way to manage when there's an update on existing database structure? How will we update the indexer without doing lots of work by creating the indexers every time?
Is there anyway we can automatically update the index whenever there is change in database records.
1 Reply
- Azure Cognitive Search can import, analyze, and index data from multiple data sources into a single consolidated search index.
Azure Cognitive search can handle large volumes of data quickly and easily, and offers a number of configurable features to allow you to tailor the search experience as per customer requirements. Azure’s Cognitive search is a powerful and reliable tool that can help you find what you’re looking for quickly and easily.
You can consider leveraging Azure Data Factory that supports PostgreSQL as source and use a storage account with a blob indexer or use Azure cognitive search index sink.
ElasticSearch/ELK is an open source search engine that is designed for horizontal scaling and distributed deployments which leverages a NoSQL database.
