To create a Custom Connector(data source) for Microsoft Purview Data Map

Question

I'm working with Microsoft Purview Data Map and I would like to create a custom data source connector for our company (let's call it Mycompany-connector). This connector would allow us to easily access and integrate data from our specific data source (e.g.custom database) within Purview.

I understand there are pre-built connectors available, but I have not found one that directly supports. As an alternative, I have considered using ETL tools to extract, transform, and load the data, but I'm hoping to explore the custom connector approach.

My questions are:
1.Is it possible to create and register a custom data source connector for Microsoft Purview Data Map?
2.If so, can you provide a detailed guide on how to achieve this?
**
I have explored alternative approaches such as:
Utilizing ETL tools like Azure Data Factory to transform data into a supported Purview format.

However, creating a custom connector seems like the most efficient solution for our specific needs.

Any guidance on building and registering custom data source connectors in Purview Data Map would be greatly appreciated.

Thanks in advance

Answer 1

@Chethan Parashuram - Thanks for the question and using MS Q&A platform.

If your data sources are not covered by the Purview connectors, you can still scan metadata for them using the following options:

Custom connectors: You can create custom connectors using the Purview SDK to scan metadata for data sources that are not covered by the Purview connectors. With custom connectors, you can define the metadata schema for your data sources and extract metadata from them.

REST API: You can use the Purview REST API to scan metadata for data sources that are not covered by the Purview connectors. The REST API allows you to create custom metadata definitions and extract metadata from your data sources.

File import: You can import metadata from files using the Purview portal. You can create a metadata schema for your data sources and import metadata from files in CSV or Excel format.

Manual entry: You can manually enter metadata for your data sources using the Purview portal. This option is useful if you have a small number of data sources or if you want to add metadata that is not available in your data sources.

It is important to note that scanning metadata for data sources that are not covered by the Purview connectors requires more effort and resources compared to scanning metadata for supported data sources. However, with custom connectors and the Purview REST API, you can automate the process of scanning metadata for your data sources and make it more efficient.

For more details, refer to below articles:

• Purview Custom Connector Solution Accelerator
• Microsoft Purview Custom Types Tool
• Deployment of the Purview Custom Connector Solution Accelerator Base Services

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.