Azure data factory AdlsGen1 structured link service fail to read file: Method not found

Question

We create a AdlsGen1 structure stream linked service to read structure stream file. When we created the link service, clicked 'Test connection' and it succeeded, but it failed to preview stream and the error was 'Type=System.MissingMethodException,Message=Method not found: 'System.Threading.Tasks.Task`1<Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationResult> Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext.AcquireTokenAsync(System.String, Microsoft.IdentityModel.Clients.ActiveDirectory.ClientAssertionCertificate)'.,Source=Microsoft.Rest.ClientRuntime.Azure.Authentication,'.

We use service principal for authentication and use cert as the SPI secret. Specifically, we create a cert in azure key vault and bind the cert to the SPI, and input the cert name as secret name in ADF. In other linked services it works well (including ADLS gen2 structure stream linked service). But for Adls gen1, it throws exception as shown before. Could you please have a look?

Answer 1

Hi @Jing Ou
I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Ask
We create a AdlsGen1 structure stream linked service to read structure stream file. When we created the link service, clicked 'Test connection' and it succeeded, but it failed to preview stream and the error was 'Type=System.MissingMethodException,Message=Method not found: 'System.Threading.Tasks.Task`1<Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationResult> Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext.AcquireTokenAsync(System.String, Microsoft.IdentityModel.Clients.ActiveDirectory.ClientAssertionCertificate)'.,Source=Microsoft.Rest.ClientRuntime.Azure.Authentication,'.

We use service principal for authentication and use cert as the SPI secret. Specifically, we create a cert in azure key vault and bind the cert to the SPI, and input the cert name as secret name in ADF. In other linked services it works well (including ADLS gen2 structure stream linked service). But for Adls gen1, it throws exception as shown before. Could you please have a look?

Solution
I created a support ticket to ADF team and they respond to me that it's a known issue and still in investigation and no ETA for it.

Regarding to the workaround, you can use managed identity to link to Cosmos Structure Stream. Specially, create a managed identity -> grant the managed identity permission in Azure portal (at least Reader) -> in ADF, use Credential to authenticate, bind the managed identity to the Credential.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

@Roi Bing I created a support ticket to ADF team and they respond to me that it's a known issue and still in investigation and no ETA for it.

Regarding to the workaround, you can use managed identity to link to Cosmos Structure Stream. Specially, create a managed identity -> grant the managed identity permission in Azure portal (at least Reader) -> in ADF, use Credential to authenticate, bind the managed identity to the Credential.