question

StephenConnell-7420 avatar image
1 Vote"
StephenConnell-7420 asked KranthiPakala-MSFT answered

Synapse Workspace Linked Service Microsoft office inside a Managed Virtual Network or REST API

Hopefully someone will have some experience of trying similar things and can offer some help.

Background



I have an Azure Synapse Analytics Workspace inside a Managed Virtual Network with Data Exfiltration Protection enabled. Public network access has been disabled. I have a customer VNET to with Private endpoints and a Virtual Machine to connect to the workspace. This works fine and I can connect to and configure a range of services e.g.:

  • Azure SQL

  • On-premises SQL using a Self Hosted IR

  • Azure Key Safe

So I'm comfortable with the concepts in general.

I have questions about connecting to other sources.

Connections


Connections to Microsoft Office for the collection of Graph data.

I have POC on another Synapse Workspace that collects BasicDataSet_v0.Message_v0 and BasicDataSet_v0.User_v0. However when I attempt to connect to the Microsoft Office Connection as a linked service from within the Managed Virtual Network I cannot connect.

 {
     "name": "Office3651",
     "type": "Microsoft.Synapse/workspaces/linkedservices",
     "properties": {
         "annotations": [],
         "type": "Office365",
         "typeProperties": {
             "office365TenantId": "[TENANT GUID]",
             "servicePrincipalTenantId": "[TENANT GUID]",
             "servicePrincipalId": "[SERVICE PRINCIPAL GUID]",
             "encryptedCredential": "ew0KICAiVmVyc2lvbiI6ICIyMDE3LTExLTMwIiwNCiAgIlByb3RlY3Rpb25Nb2RlIjogIktleSIsDQogICJTZWNyZXRDb250ZW50VHlwZSI6ICJQbGFpbnRleHQiLA0KICAiQ3JlZGVudGlhbElkIjogIlNZTkFQU0VAQUJFNEVBODktM0E5MS00RjExLTgzOTItMjcyOURFNTY2NDk3XzhhMGViNzIyLTRmNTAtNGQ4Zi1iYTVhLTYwYWNiNzQyYjMzMiINCn0="
         },
         "connectVia": {
             "referenceName": "AutoResolveIntegrationRuntime",
             "type": "IntegrationRuntimeReference"
         }
     }
 }

The error I get attempting to validate the connection is:

 One or more errors occurred.
 An error occurred while sending the request.
 Unable to connect to the remote server
 A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 51.105.5.65:443
 Activity ID: 218a9020-6de9-4436-b61b-0e62c6b9c147.

As indicated I can get this to work without a Managed VNET.

I get similar experiences for REST Api connections, these are connections which I have used outside of a Manage VNET

 The connection to the REST service failed. Endpoint=https://***URL***/, Reason=.
 A task was canceled.
 Activity ID: 7bc36bd7-d135-4636-8303-89acc84ef711.

Request



If anyone has tried similar or has found documentation for a more detailed exploration of how to create linked services I would be very grateful. It becomes harder to justify the use of the Managed VNET if these are hard limitations.

azure-synapse-analytics
· 9
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Update

I have tested both of these approaches with out using Data Exfiltration Protection. This seems to be the blocking technology.


0 Votes 0 ·

Hello @StephenConnell-7420,

Thanks for the question and using MS Q&A platform.

My understanding is that you are trying to connect to Office 365 using Office 365 connector in Azure Synapse with Managed VNET and Data Exfiltration Protection enabled. But the test connection is failing with above mentioned error. But when you try to establish the connection to Office 365 from an Azure Synapse Workspace without Managed VNet then your test connection is successful. And you would want to know what is the limitation that is causing this behavior in Azure Synapse Workspace with Managed VNET. Please correct my understanding is not accurate.

I have also noticed that you have posted a similar ask in Stackoverflow here: Linked Service Microsoft office inside a Managed Virtual Network or REST API.

172062-image.png

From the stackoverflow thread my understanding is that you are able to establish a connection to Office 365 from a Synapse Workspace with Managed VNET but not with Synapse Workspace with Managed VNET and Data Exfiltration Protection enabled. So, the problem is only when with Data Exfiltration Protection enabled. Please correct if my understanding is incorrect.

Could you please tell us what is you Synapse workspace (with Managed Vnet + Data exfiltration) region and your Office 365 tenant region to which your connection is failing?




0 Votes 0 ·
image.png (9.6 KiB)

Hi Kranthi.
Your understanding is correct. I can use O365 & API with Managed VNET without Exfiltration Protection.
My set up is a bit convoluted as I have provisioned a trial Office 365 tenant along with a trial Azure in order to test some Graph implementation for a customer but my Synapse implementation is within my MPN subscription.

The Synapse with Managed Vnet + Data exfiltration is /subscriptions/d2fd58d1-25f7-4ace-a06f-2b702070b971/resourceGroups/fullsecuremode/providers/Microsoft.Synapse/workspaces/fullsecuremode
The Synapse without Exfiltration protection is:/subscriptions/d2fd58d1-25f7-4ace-a06f-2b702070b971/resourceGroups/ManagedVnet/providers/Microsoft.Synapse/workspaces/managedvnet

Due to graph restrictions I am extracting Graph data to the same tenant. As such my tenant ID for Graph data is 3c38a94b-4d94-4e27-a6ae-57794c94f80a and the storage I use for extractions is /subscriptions/99685c20-a9ca-4b5f-9bc2-578663be5940/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Storage/storageAccounts/graphimportadsl/blobServices/default.

Due to the limits of the trial account I cannot create a synapse with Managed VNET on the trial and due to MPN restrictions all of my implementation are in West Europe.

I am happy with the state of things for now. We will recommend to clients with API, Graph data or OData for external web applications (not Azure) not to enable Data Exfiltration however I can't imagine that this is not what was expected to happen in this scenario so will be keeping an eye on developments.
I hope this is enough detail.
Stephen.

0 Votes 0 ·

Hello @StephenConnell-7420,

Appreciate much for the detailed explanation. I did reach out to internal teams regarding this issue, and we have suggestions that this need to have a deeper investigation by support team. I would recommend filing a support ticket so that the support engineer can involve relevant product owners to have deeper analysis. Please do share the SR# with us once it is opened so that we can track it internally. Incase if you don't have a support plan do let us know here so that we can work offline on the next steps.

Thank you

0 Votes 0 ·

Could you provide some detail of where to raise this support ticket? As I say my environments are split over trials and MPN subscriptions. I do have access to the partner center https://partner.microsoft.com/en-us/dashboard/support/servicerequests. Would this be the best place?

0 Votes 0 ·

Hi Kranthi. I have tried to raise a support ticket from Azure but it turns out we only have basic support. Is there some other route we can take?

0 Votes 0 ·
Show more comments
StephenConnell-7420 avatar image
0 Votes"
StephenConnell-7420 answered 35247258 converted comment to answer

I have with support from the team here managed to get a partial working solution.
For rest API, OData, and a lot of other services too numerous to mention here; the solution is to set up a Self Hosted Integration Runtime (SHIR). This SHIR has a node on a virtual machine which sits in a Custom VNET with Data Link and Private Endpoints. This then allows the connection of these services using the SHIR rather than an Azure Auto Resolve Integration Runtime.

While this allows for many services to connect it does not allow Office 365 connections as this will not permit connections using SHIR. The only work around is to not use Microsoft Graph Data Connect but use the API and SHIR or not deploy Data Exfiltration Protection. I am informed it is in the plan to fix this but don't have details on the time frame for this resolution.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

176815-2022-02-22-13-30-24-private-end-pointsvsdx-visio-p.png


Hopefully this illustrates the approach which was suggested. While not documented it appears that Workspaces with data exfiltration enable will block access to all public endpoints.

1 Vote 1 ·
KranthiPakala-MSFT avatar image
2 Votes"
KranthiPakala-MSFT answered

Hello @StephenConnell-7420,

Thanks for sharing update from support request. As it turns out at this time the product feature you are looking is not supported. May I request you to please log your feedback in IDEAS forum here: IDEAS forum.

Product group does monitor the request and they can plan for the implementation in future. Once you log the feature request you will also be notified on the status of the request. Please do share your the link once the feedback is posted as it would help others up-vote and comment on it to increase the priority.

Thanks
Kranthi


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.