Azure Digital Twins Import Jobs Api - import job fails when property name is "$componentName.$fieldName"

marcin.rejdych 20 Reputation points
2023-05-26T13:47:29.76+00:00

Details of our use case:

  • We want to implement our Azure Digital Twins Backup and Restore procedure by using Azure Digital Twins Import Jobs API and NDJSON files.
  • Our Digital Twin's Model have few Components declared
  • We are using ADT Data History to fill 3 ADX tables with data
    • Table with Twin lifecycle events
    • Table with Twin properties update events
    • Table with Relationship lifecycle events
  • We are using data from these ADX tables to create NDJSON file, which will later be used to restore state of Azure Digital Twin
  • During exporting of NDJSON file to Blob Storage we do not have access to information about Digital Twin Models
    • Our NDJSON file contains sections: "Header", "Twins", and "Relationships" (no "Models")
    • We intend to import our Digital Twins Models using different method (before Import Jobs API - Add will be executed)

Description of our problem:

When single property of Digital Twin's Component is updated, then Twin's Property Update Event is stored in ADX table.

Column "Key" in ADX table contains value, which have following format - "componentName.updatedPropertyName".

We are generating NDJSON file from data stored in ADX by ADT Data History in order to be used with ADT Import Jobs API and we would like to use as simple KQL as possible.

When we want to import NDJSON file using Import Jobs - Add API to restore state of Azure Digital Twins, job is failing with following error:


{\"code\":\"ValidationFailed\",\"message\":\"componentName.updatedPropertyName does not appear on the DTDL for Twin.\",\"details\":null}

Workaround:

We had to add additional complexity to Kusto Query to aggregate updates of single properties with Key componentName.updatedPropertyName into JSON object, which represents Component - for example:

   "componentName": {
     "updatedPropertyName": $Value,
     "$metadata": {}
   }

Question:

Considering the fact, that Import Jobs API is currently in Preview, could you consider feedback to change its behaviour to be able to handle partial updates of Twin's Components (when for example property have name "componentName.updatedPropertyName") as it would be compatible with a way, how Twin's Property Update Event is stored in ADX using ADT Data History feature?

If it is not possible, then do you have any advice to handle it in a better way than aggregating such properties into Component JSON Object using KQL?

Thank you for your help!

Azure Digital Twins
Azure Digital Twins
An Azure platform that is used to create digital representations of real-world things, places, business processes, and people.
219 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. LeelaRajeshSayana-MSFT 13,456 Reputation points
    2023-05-31T13:02:52.63+00:00

    Hi @marcin.rejdych Thank you for sharing your insights on this product.

    You can share product feedback on this feature using the following link Azure Digital Twins Feature Request Kindly share the link on this thread once you have created a request. Other community members needing a similar feature can help upvote it.

    I am also checking with the product team internally on any alternate better approaches we can take to address this. I will update this thread once I have more information.

    If the response helped, please do click Accept Answer and Yes. Doing so would help other community members with similar issue identify the solution. I highly appreciate your contribution to the community.

    0 comments