Without Purview, how to add metadata tags to Integration Datasets in Azure with lineage?

Question

Without Purview, how to add metadata tags to Integration Datasets in Azure with lineage?

Christopher Lindeman 0

It looks as though we won't get Purview before we need to start tagging data. The data is being tagged with a security Classification 1-5. The purpose is that when data is merged/joined/etc. that the highest security classification travels along with the new view, ensuring that security protocols are maintained.

PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-07-03T09:31:47.32+00:00

@Chris Lindeman - Following up to see if the below answer was helpful.
PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-07-06T08:34:40.8+00:00

@Chris Lindeman - Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Your answer

PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-07-03T09:31:47.32+00:00

@Chris Lindeman - Following up to see if the below answer was helpful.
PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-07-06T08:34:40.8+00:00

@Chris Lindeman - Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

PRADEEPCHEEKATLA 91,496 Moderator

@Anonymous - Thanks for the question and using MS Q&A platform.

Note that without Purview, you may need to manually track the lineage of the data and ensure that the security classification is maintained. This can be a time-consuming and error-prone process, especially for large and complex datasets.

I hope this helps! Let me know if you have any further questions.

PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-07-07T06:06:25.7566667+00:00

@Chris Lindeman - Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Christopher Lindeman 0 Reputation points

2023-07-07T14:18:58.9933333+00:00

Can you describe a process to manually track the tagged metadata parameter? I am using our chat to convince management to get Purview for us as I imagine this workaround will be clunky and difficult.
PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-07-13T04:17:50.9666667+00:00
@Chris Lindeman - Manually tracking metadata tags for Integration Datasets in Azure can be a time-consuming and error-prone process, but it is possible to do so using a combination of documentation and naming conventions. Here are the steps to manually track metadata tags:

Create a document that lists all the Integration Datasets in your Azure Data Factory environment and their associated metadata tags. This document should include the name of the Integration Dataset, the source and destination data sources, and the metadata tags associated with the Integration Dataset.

Use a naming convention for the Integration Datasets that includes the metadata tags. For example, you could include the security classification in the name of the Integration Dataset, such as "SalesData-SecurityClass-3". This will make it easier to identify the metadata tags associated with each Integration Dataset.

Use a consistent naming convention for the metadata tags. For example, you could use a prefix such as "SecurityClass-" followed by the security classification number. This will make it easier to search for and filter Integration Datasets based on their metadata tags.

Update the document and naming conventions as needed when new Integration Datasets are added or when metadata tags are changed.

Use the document and naming conventions to track the metadata tags associated with each Integration Dataset. This can be done by searching for Integration Datasets based on their metadata tags or by manually reviewing the document.

Note that this approach requires manual tracking and is prone to errors and inconsistencies. It is also difficult to scale and maintain as the number of Integration Datasets and metadata tags increases. Purview provides a more automated and scalable solution for metadata management and lineage tracking.
PRADEEPCHEEKATLA 91,496 Reputation points Moderator

2023-07-17T08:08:36.8133333+00:00

@Chris Lindeman - Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

Without Purview, how to add metadata tags to Integration Datasets in Azure with lineage?

1 answer

Your answer