SharePoint Online and Azure Data Fabric

Question

SharePoint Online and Azure Data Fabric

Blair Lawrence 21

Is there an architectural reference to map SPO/M365 data into Azure Data Fabric without duplicating data?

Renjie Sun-MSFT 2,846 Reputation points Microsoft Employee

2022-10-27T06:23:36.58+00:00

Hi @Blair Lawrence ,

As to your issue, it is more related to Azure Data. I will remove the office-sharepoint-online tag from this case, so that you could get more valid help.

Regards,
BhargavaGunnam-MSFT 21,336 Reputation points Microsoft Employee

2022-11-04T22:38:28.667+00:00

Hello @Renjie Sun-MSFT ,
Sorry, this question doesn't belong to the data factory or data bricks. I have updated the office-SharePoint-online tag for now.

Hi @Blair Lawrence ,
Could you please provide more details about the ask? So that we can route this question to the right team?
Thank you for your patience.
Blair Lawrence 21 Reputation points

2022-11-05T16:33:40.407+00:00

I remove the comment.
Blair Lawrence 21 Reputation points

2022-11-05T16:34:52.143+00:00

Is there a method that supports pointers from within Databricks to original data sources and not have to duplicate data. Hybrid Example: I have data hosted within the MS ODSP environment and also other on prem data sources such as MSSQL and file shares and wish to centrally manage/control/collect telemetry on that data using structure and control surfaces similar to Databricks. With the current architecture I have to duplicate data and move it into Databricks to control the data.
BhargavaGunnam-MSFT 21,336 Reputation points Microsoft Employee

2022-11-07T22:24:25.043+00:00

Hi @Blair Lawrence ,
Thanks for the reply.
You have Microsoft One drive, Share point, MSSQL, and file shares, and you wanted to control/collect telemetry from these systems. Are you currently using Azure data bricks to collect/control the data?
Could you please provide more details about "move it into Databricks to control the data."
Are you looking for any data warehouse solution here?
Blair Lawrence 21 Reputation points

2022-11-08T13:53:37.47+00:00

Thank you for the reply.
There are currently no "central" automated controls (such as drillable dashboards) for data and unified data governance in the enterprise. To design a unified data fabric architecture that supports Microsoft ODSP, "as well as" other hybrid data sources such as MS SQL, file shares, and so on, the data fabric architecture begins to increase in complexity, support, and administrative overhead costs. Microsoft Purview would have been ideal, but it does not support anything in the MS ODSP collective except Power BI. I am looking at Databricks as a possible solution for enterprise-wide unified data governance, but I am not seeing any value or practical need to duplicate data into storage systems for this need when a unified pointer system would be more cost effective.

Renjie Sun-MSFT 2,846 Reputation points Microsoft Employee

2022-10-27T06:23:36.58+00:00

Hi @Blair Lawrence ,

As to your issue, it is more related to Azure Data. I will remove the office-sharepoint-online tag from this case, so that you could get more valid help.

Regards,
BhargavaGunnam-MSFT 21,336 Reputation points Microsoft Employee

2022-11-04T22:38:28.667+00:00

Hello @Renjie Sun-MSFT ,
Sorry, this question doesn't belong to the data factory or data bricks. I have updated the office-SharePoint-online tag for now.

Hi @Blair Lawrence ,
Could you please provide more details about the ask? So that we can route this question to the right team?
Thank you for your patience.
Blair Lawrence 21 Reputation points

2022-11-05T16:33:40.407+00:00

I remove the comment.
Blair Lawrence 21 Reputation points

2022-11-05T16:34:52.143+00:00

Is there a method that supports pointers from within Databricks to original data sources and not have to duplicate data. Hybrid Example: I have data hosted within the MS ODSP environment and also other on prem data sources such as MSSQL and file shares and wish to centrally manage/control/collect telemetry on that data using structure and control surfaces similar to Databricks. With the current architecture I have to duplicate data and move it into Databricks to control the data.
BhargavaGunnam-MSFT 21,336 Reputation points Microsoft Employee

2022-11-07T22:24:25.043+00:00

Hi @Blair Lawrence ,
Thanks for the reply.
You have Microsoft One drive, Share point, MSSQL, and file shares, and you wanted to control/collect telemetry from these systems. Are you currently using Azure data bricks to collect/control the data?
Could you please provide more details about "move it into Databricks to control the data."
Are you looking for any data warehouse solution here?
Blair Lawrence 21 Reputation points

2022-11-08T13:53:37.47+00:00

Thank you for the reply.
There are currently no "central" automated controls (such as drillable dashboards) for data and unified data governance in the enterprise. To design a unified data fabric architecture that supports Microsoft ODSP, "as well as" other hybrid data sources such as MS SQL, file shares, and so on, the data fabric architecture begins to increase in complexity, support, and administrative overhead costs. Microsoft Purview would have been ideal, but it does not support anything in the MS ODSP collective except Power BI. I am looking at Databricks as a possible solution for enterprise-wide unified data governance, but I am not seeing any value or practical need to duplicate data into storage systems for this need when a unified pointer system would be more cost effective.

Answer 1

BhargavaGunnam-MSFT 21,336 Microsoft Employee

Hi @Blair Lawrence ,
Sorry for the delayed response.
I discussed with my internal team on this. Per the advice, Microsoft Purview would be the ideal solution for your requirement. Since MS ODSP is not supported in purview, you may move ODSP to one of the purview-supported data sources for unified data governance.
I hope this helps.

Blair Lawrence 21 Reputation points

2022-11-16T19:44:26.897+00:00

Perhaps there is a misunderstanding. MS Purview cannot connect to SharePoint Online (SPO). How would you simplify and centralize enterprise-wide data architecture to support data governance and maturity for "all enterprise data sources" including SPO?
BhargavaGunnam-MSFT 21,336 Reputation points Microsoft Employee

2022-11-16T20:24:14.297+00:00

Hi @Blair Lawrence ,
Correct, MS Purview cannot connect to SPO. You will need an additional solution to move SPO data to Purview-supported data sources.
Example: You can use logic apps to move SPO data to Azure blob storage. Azure blob storage is a supported data source for MS Purview.

SharePoint Online and Azure Data Fabric

1 answer

Activity