Event banner
Event details
We have implemented azure cognitive search for finding products based on their attributes and this works very well. Now a customer has the requirement to be able to search products based on the ETIM classification attributes. This classification has a predefined number of over 14.000 unique attributes accross all product categories. As each unique attribute will consume at least one simple field we run into the index limit of a 1000. Maybe we have chosen the wrong architecture, but what is the right one to use when we have such a requirement?
MicrosoftThis appears to be a question about the 1000 field limit in Azure Cognitive Search. There isn't an easy workaround to the field limit, but you do have some options to get the experience you are looking for.
These are 3 options I would suggest considering:
- Field reuse within an index. If each product catalog entry would use less than 1000 fields, you could reuse the same fields for multiple product types with your application managing the mapping of the meaning of each attribute to the underlying field in the index. This option does have drawbacks since each field would have the same settings across multiple product types.
- Extend catalog to multiple indexes. You could effectively store your catalog in multiple indexes where the full set of fields will extend across the indexes. This would work best if an individual catalog entry doesn't have more than 1000 fields again and would run into some cross-index query issues. But you should be able to get faceting to work in this option.
-
Attribute metadata fields. Define a few general fields in your index to store attribute metadata. These fields could include "attribute_name," "attribute_value," and "product_id" or any other relevant identifiers. For each product, store its ETIM classification attributes as separate records in the attribute metadata fields. This means you'll have multiple attribute records for each product, one for each ETIM attribute associated with that product. To implement this solution, you'll need to update your data indexing and search query pipelines to work with the attribute metadata fields and apply filters accordingly. It may require some adjustments to your existing code, but it should provide a way to handle the complex attribute requirements.