Azure AI Document Intelligence
An Azure service that turns documents into usable data. Previously known as Azure Form Recognizer.
How to set the language when training a custom template model
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 67%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJP%3C/text%3E%3C/svg%3E)
Jonathan Perry
0
Reputation points
Hello,
I'm trying to train a custom model to recognize template of forms in hebrew, as the pretrained models of invoices etc do not support hebrew.
The auto labeling process does not seem to recognize that it is in hebrew, although hebrew is read correctly when doing a generic read model.
Is there a way to force the custom template model to use hebrew?
Thanks.
Azure AI Document Intelligence
{count} votes
-
romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator2023-05-23T12:18:32.7733333+00:00 @Jonathan Perry I think the build mode or parameters to pass for training do not take the language code since Form Recognizer's deep learning based universal models use all multi-lingual text in your documents, including text lines with mixed languages, and do not require specifying a language code. The same applies during extraction process and the language code is optional for analyze operation. However, the 2023-02-28-preview version of the API supports Hebrew and if you use this version of the API to build the custom model the extraction process should also recognize the same since it is in the list of supported languages. While using the studio, creation of new project should allow you to select this API version to start training a custom model and then you can call its corresponding analyze API to extract text. I hope this helps!!
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 27%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
VasimTamboli 5,535 Reputation points MVP2023-05-23T13:02:36.95+00:00 if i understood correctly -
When training a custom template model in Azure Form Recognizer, you can specify the language of the documents you're training on, including Hebrew. By setting the language appropriately, you can train the model to recognize and process Hebrew documents more accurately. Here's how you can do it:
Prepare your training data: Gather a dataset of Hebrew forms or documents that you want to use for training. Make sure the documents are representative of the type of forms you expect the model to process.
Use the Form Recognizer SDK or API: Utilize the Form Recognizer SDK or API to interact with the service and train your custom template model.
Set the language parameter: When training the model, specify the language parameter as "he". This informs the model that the training data contains Hebrew documents.
Here's an example of how to set the language parameter using the Form Recognizer Python SDK:
pythonCopy code from azure.ai.formrecognizer import FormTrainingClient from azure.core.credentials import AzureKeyCredential # Create an instance of the FormTrainingClient endpoint = "https://<your-form-recognizer-endpoint>" credential = AzureKeyCredential("<your-form-recognizer-key>") form_training_client = FormTrainingClient(endpoint, credential) # Specify the language when training the model training_data_url = "<your-training-data-url>" model = form_training_client.begin_training(training_data_url, model_language="he").result() # Wait for the training process to complete model.wait_until_done() # Retrieve the trained model ID model_id = model.model_idBy setting the
model_languageparameter to "he", you inform the model to consider the training data as Hebrew documents.Note that the effectiveness of the model's recognition will depend on the quality and variety of the training data. Make sure to provide a diverse and representative set of Hebrew documents to achieve better results.
Once the model is trained, you can use it to extract information from Hebrew forms using the recognized template.
-
Jonathan Perry 0 Reputation points
2023-05-24T22:04:41.34+00:00 @romungi-MSFT I am using the studio custom extraction model.
In the labeling process, I'm trying to label the field name, but it is recognized as latin characters.
I didn't find any place to configure the language of the file/document.
Is there any way to improve this? I think the value of the field will be extracted better if I label the field key correctly.
-
romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator
2023-05-25T06:58:23.1+00:00 @Jonathan Perry Yes, I think hebrew is still not being recognized. It is unable to distinguish it from latin script. I have tried with another language, hindi and the studio seems to recognize devnagari script with the preview API of the project.
Since there is no option to set the language, it seems like the API is yet to add support to recognize Hebrew.
Sign in to comment
Sign in to answer