question

RajD-9527 avatar image
0 Votes"
RajD-9527 asked MartinJaffer-MSFT commented

Upload files to azure data lake storage container or file share using SSIS 2019 script task

Hi, I am looking at different ways to save file using SSIS 2019 script task to Azure data lake store gen2. I make a call to a REST api and save the xml payload on to an Azure data lake store gen2 and looking at options to connect to an Azure tenant and save files to the storage account. I am using the below C# code in my script task. The issue I'm running into is I cannot read directly from the memory stream and write data to either the container or the file share. How to connect to the azure container or file share using SAS token?

Code:

 using Microsoft.Azure.Storage.Auth;
 using Microsoft.Azure.Storage.Blob;
 using System;
    
 namespace AzureStorageTest
 {
     class Program
     {
         static void Main(string[] args)
         {
    
             string storageAccountName = "<storage account name>";
             string containerName = "<container name>";
             string sasToken = "<sas token>";
             StorageCredentials creds;
             CloudBlobContainer cloudBlobContainer;
             creds = new StorageCredentials(sasToken);    
             cloudBlobContainer = new CloudBlobContainer(new Uri("https://"+ storageAccountName + ".blob.core.windows.net/"+ containerName), creds);
             BlobContinuationToken blobContinuationToken = null;
             var blobs = cloudBlobContainer.ListBlobsSegmented("", blobContinuationToken);
             foreach (var blob in blobs.Results) {
                 MessageBox.Show("blob.Uri");
             }
         }
     }
 }

Thank you in advance.

dotnet-csharpsql-server-integration-servicesazure-data-lake-storage
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @RajD-9527,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if have already reached out to the Microsoft Support, please do share the SR# , otherwise request you to please do the needful as requested in the last response .

Thanks
Martin

0 Votes 0 ·

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered

Hello @RajD-9527 and welcome to Microsoft Q&A.

If I understand correctly, you want to upload blobs using C# .Net SDK, but aren't sure what call to use.

First, a few clarifications:
The difference between Azure Storage Blob, and Azure Data Lake Gen 2, is Hierarchical Namespace. When enabled, it is known as Data Lake Gen2. When disabled, it is Blob. While, Data Lake Gen 2 is built on top of Blob, they have different API's. Using the wrong one can end up with a file named "myFolder/myFile.txt" instead of a file by the name of "myFile.txt" in a directory named "myFolder".

File Share is yet another different storage service.

For Data Lake Gen2, I recommend looking link here.

SAS example link here at line 116.

There is more than one type of blob (Applend, Block, Page, etc.). Each type has a different class. That is why you couldn't find upload method in the CloudBlobContainer class. Example, UploadBlockBlobFromFile link.


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.