How to connect to a folder in Azure Virtual Machine with Azure Data Factory

King Java 500 Reputation points
2023-11-15T23:50:59.9333333+00:00

I am trying to use Azure Data Factory to move csv files from Azure Storage (File service) to a folder in Azure Virtual Machine (on Prem).

How do I go about creating Linked Service?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,198 questions
{count} votes

2 answers

Sort by: Most helpful
  1. Amira Bedhiafi 20,176 Reputation points
    2023-11-16T10:02:41.4166667+00:00

    For creating the Linked Service for Azure VM , you need to install the Self-hosted Integration Runtime on your Azure VM. This acts as a bridge between Data Factory and on-premises data stores.

    In Data Factory Studio, choose a relevant on-premises data store type (for example File System if it's a simple folder).

    https://learn.microsoft.com/en-us/azure/data-factory/create-self-hosted-integration-runtime?tabs=data-factory

    1 person found this answer helpful.
    0 comments No comments

  2. PRADEEPCHEEKATLA-MSFT 85,746 Reputation points Microsoft Employee
    2023-11-16T10:12:01.4966667+00:00

    @King Java - Thanks for the question and using MS Q&A platform.

    What do you mean by Azure Virtual Machine (On Prem)? Are you referring to on-premise windows server?

    FYI, Azure Virtual Machines are typically hosted in the cloud.

    If you are looking for a solution to connect to folder in Azure Virtual Machine which is hosted on Azure, you can Create an SMB Azure file share and connect it to a Windows VM using the Azure portal. Azure Files networking considerations | Microsoft Learn

    Once the SMB mounted you can move the files to the windows folders to Azure Files share with ease.User's image

    Azure Files offers fully managed file shares in the cloud that are accessible via the industry standard Server Message Block (SMB) protocol or Network File System (NFS) protocol.

    For more details, refer to Tutorial: Create an SMB Azure file share and connect it to a Windows VM using the Azure portal and Mount SMB Azure file share on Windows.

    If you are looking for a solution to connect to folder in window servers which is on-premise you can use Azure Data factory to connect via file system connector using SHIR. User's image

    Azure Data Factory file system connector supported for the following capabilities:

    User's image

    • Copying files from/to network file share. To use a Linux file share, install Samba on your Linux server.
    • Copying files using Windows authentication.
    • Copying files as-is or parsing/generating files with the supported file formats and compression codecs.

    For more details, refer to Copy data to or from a file system by using Azure Data Factory or Azure Synapse Analytics.

    Hope this helps. Do let us know if you any further queries.


    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.