Connecting and running Linux and unix scripts in on-Prem Linux servers from Azure data factory pipelines

Question

Connecting and running Linux and unix scripts in on-Prem Linux servers from Azure data factory pipelines

Firthouse M G 80

Hello Team, This is regarding the connection of on-Prem Linux servers and running scripts and creating and accessing files from Azure data factory for ETL process. We have created a VM in azure to install SHIR to connect to the On-Prem Oracle servers. Is there a way to connect on-Prem linux servers and run scripts with the same SHIR? What could be the correct way to do this?

phemanth 15,755 Reputation points Microsoft External Staff Moderator

2024-01-08T03:16:11.91+00:00

@Firthouse M G We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
phemanth 15,755 Reputation points Microsoft External Staff Moderator

2024-01-09T05:16:23.89+00:00

@Firthouse M G just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Firthouse M G 80 Reputation points

2024-01-23T15:30:21.7+00:00
@phemanth Successfully installed SHIR. Thank you for your assistance!

Accepted answer

0 additional answers

Your answer

phemanth 15,755 Reputation points Microsoft External Staff Moderator

2024-01-08T03:16:11.91+00:00

@Firthouse M G We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
phemanth 15,755 Reputation points Microsoft External Staff Moderator

2024-01-09T05:16:23.89+00:00

@Firthouse M G just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Firthouse M G 80 Reputation points

2024-01-23T15:30:21.7+00:00

@phemanth Successfully installed SHIR. Thank you for your assistance!

Answer 1

@Firthouse M G

Thanks for reaching MS Q&A

Yes, you can use a single self-hosted integration runtime (SHIR) for multiple on-premises data sources. The SHIR server does not have to be on the same server as your data source, but it should be close to it. Here are some steps you can follow:

Install SHIR: Install the self-hosted integration runtime on an on-premises machine or a virtual machine inside a private network. https://learn.microsoft.com/en-us/azure/data-factory/create-self-hosted-integration-runtime?tabs=data-factory
Configure SHIR: After installation, configure the SHIR to connect to your on-premises Linux servers.
Run Scripts: For running scripts on your Azure VM, you can use the Run Command feature. You can also download and run scripts in Azure virtual machines using Azure Resource Manager templates, Azure CLI, REST API, PowerShell, or Azure portal. https://learn.microsoft.com/en-us/azure/data-factory/create-self-hosted-integration-runtime?tabs=data-factory
Run Scripts: For running scripts on your Azure VM, you can use the Run Command feature3. You can also download and run scripts in Azure virtual machines using Azure Resource Manager templates, Azure CLI, REST API, PowerShell, or Azure portal. https://learn.microsoft.com/en-us/azure/virtual-machines/linux/run-scripts-in-vm

Remember, the SHIR doesn’t need to be on the same machine as the data source. However, having the SHIR close to the data source reduces the time for the SHIR to connect to the data source. We recommend that you install the SHIR on a machine that differs from the one that hosts the on-premises data source.

Hope this helps. Do let us know if you any further queries.

Share via

Connecting and running Linux and unix scripts in on-Prem Linux servers from Azure data factory pipelines

0 additional answers

Your answer