How to extract columns metadata from Azure Data Catalog api and register asset

Question

I am trying to register a SQL data table in Azure data catalog through REST API. It is getting registered but I have to manually include any details like Columns/Schema or Preview. I used one property “fromSourceSystem" : true in json body to make it dynamic means it should pick schema/columns/preview from data source automatically but I am not able to get it working. Also, is it possible to populate data profiling automatically through REST Api ?

Answer 1

Hi anonymous user,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

As per documentation, fromSourceSystem is an boolean indicator that signifies whether data is derived from source system or authored by the user and cannot populate the columns from the source system. I could not find any workaround for skipping passing the columns information using REST API while registering the data source.

You can use the GET API for the asset and it will retrieve the data profile as well. There is no specific API to do the same.

I would recommend you to submit ideas for the above in feedback forum. All the ideas posted in this forum are closely monitored by data catalog product team and will implement in future releases.

Hope this helps! Please let us know for further queries and we will be glad to assist.