Azure AI Search token filter equivalent to Multiplexer in ElasticSearch

Question

Hi there,

Do we have the same token filter "Multiplexer" as ElasticSearch?

A token filter of type multiplexer will emit multiple tokens at the same position

https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-multiplexer-tokenfilter.html

Thanks.

Answer 1

@Jay Lin Thanks for reply! Assuming the content is in a specific language, you could use the Microsoft analyzers (e.g. en.microsoft), which normalize numbers and dates.

That would handle the thousand separator case, trailing zeroes after the decimal separator, and other variations that don't alter the number.

From your list of examples, the one that wouldn't be addressed is 1234.56 -> 1234 because that's a lossy conversion.

Check this document- Add language analyzers to string fields - Azure AI Search | Microsoft Learn. Using the /analyze endpoint in indexes is a great way to experiment and see how analyzers handle and tokenize different inputs.

Hope this helps. Let me know

Answer 2

@Jay Lin Thanks for asking question!

Could you elaborate on if any specific scenario you are looking for, or are you simply looking for an equivalent feature?

In Azure Search we don't have a like to like token filter- here are the token filters available in Azure Search. I didn't see any multiplexer among them: https://learn.microsoft.com/en-us/azure/search/index-add-custom-analyzers#token-filters.

Please let us know, happy to assist further.