Pashto language understanding (written in English alphabets)

Question

Hello everyone! I hope you all are well! I am using Azure cognitive service , language studio for language understanding. And I want to design a model which can detect specific type of content in a text. in Pakistan we write Pashto in English alphabets for messaging and on social media just like people in India writes Hindi in English alphabets. So I want to detect certain type of content in a text of Pashto langauge (written in English letter) so how can I do this? Do anyone of you know that how azure Langauge studio train a model is it only looks to the structure of langauge or it also take the meaning of text?
like How are you in Pashto is written as
ته څنګه یاست

But if I write this pashto sentence in English alphabets then it will be
"Ta sanga ye" Now this text has no meaning in English but a pashto speaker can understand its meaning so this is pashto language sentence written in English and I want to train a model on such type of text.
so what should I select for language section it will be pashto or english? I want to know that language studio also considers the meaning or it only classify on the basis of structure?
it is just like Hindi and urdu , both are same in speaking but different in writting.

Answer 1

@Anonymous Azure translator service has an API for transliterate where one script is changed to another without changing the pronunciation in English. However, this API does not support Pashto for transliteration. Other languages like Arabic, Hindi, Punjabi & Urdu are supported. This might work with some scenarios with Urdu but not completely.

The Language studio feature to train a model using conversational language understanding(CLU) supports Pastho language but not with text written in English (Latin script). You will have to use Pastho if a project is created with Pastho.

Using English as project's language to train a model for Pashto will also not work because the base models are trained in English and Pastho text in English will not be recognized/trained correctly.

Answer 2

I do research on the same topic in the name of Pashto - English sentiment analysis which I need data to train the module and test its accuracy and to be known machines what we are writing and categorize our opinions.