Forum Discussion
Search integration with ETL/DW/Synapse(DL)
We're looking at implementing an enterprise search, which includes a revamp of our EDP (enterprise data platform). Creating an etl which will ingest data from various sources via ADF into a DL to be ...
liamca-msft
Microsoft
MicrosoftNatural Language (NLP) is a really broad area, so what I would suggest is to break down what is actually need by the user. For example, sometimes the goal is to be able to use NLP to ask questions and get answers. Or othertimes, it is to be able to let users enter a search query that is not a set of words, but more like a phrase and have the search understand it. For example, when I search "hot dog" it should know that I am not referring to "puppies", but rather food. For each of these cases, I highly recommend that you take a look at the semantic search capability we just introduced (https://docs.microsoft.com/en-us/azure/search/semantic-search-overview). In some cases, there are really advanced NLP cases. For example "find me all the homes in seattle that have 3 bedrooms and are less than $1m". This is advanced because you need to understand that some of the query ("3 bedrooms") need to be mapped to a filter query (bedrooms == 3). for this, you can do training of the users intent using LUIS (https://www.luis.ai/), but this requires a lot of training, and I would not likely suggest that you start here.
But if we have data that contains information about people connected to a particular project (via timesheets, CRM data, etc) and we ask it to give me a list of people working on project XXXX would semantic search be a good option for that or LUIS need to be brought in?
- liamca-msft
aowens-jmt that is a great example. You could definitely use LUIS to help here because you could break down the intent of this query into a search where project == XXXX. In this particular case, I don't think Semantic Search would help because if you sent a query of "give me a list of people working on project XXXX", the only term in this that would actually help find relevant content in the search index is XXXX. It would not know that the user wants to limit search to the field "project". It is not to say that you need to use LUIS, and you could certainly have you web app intercept this search query and rewrite it as a search query (search=*&$filter=project eq XXXX).