Within the dimension properties for an attribute within SSAS, there is an option called TokenizationBehavior which is defaulted to TokenizationNone. The alternative is TokenizationText.
I tried it out and it gives an error message as per below. There is little content on the internet about anybody trying this out or what it is actually for. Any ideas? I'm not sure if this Tokenization is for parsing text or a security mechanism? And why it doesn't work or how it's supposed to work in SSAS?
The ddl500:TokenizationBehavior element (namespacehttp://schemas.microsoft.com/analysisservices/2013/engine/500) cannot appear under Envelope/Body/Execute/Command/Batch/Alter/ObjectDefinition/Database/Dimensions/Dimension/Attributes/Attribute
I looked at the XML code which adds the below line if you put TokenizationText in the dimension attribute properties: