Running SHIR in Window Container

Question

Running SHIR in Window Container

Meyliana Wangsa 5

Currently have an integrated runtime for Azure synapse workspace running on a Windows VM. It seems like a waste of resources and more expensive (paying for the windows license, managing the VM and so on) versus running the service in a container. Has anyone tried running the self-hosted integration runtime (SHIR) in a windows container? https://docs.microsoft.com/en-us/azure/data-factory/how-to-run-self-hosted-integration-runtime-in-windows-container

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-04-06T01:13:41.3433333+00:00

Hi @Anonymous ,

Thanks for using Microsoft Q&A forum and posting your query.

Checking to see if you are facing any issue while following the steps provided in the documentation. If so, kindly elaborate a bit on what was the error message, and at which step you are noticing an issue so that we can assist you accordingly.

Or if you notice any steps that missing in the documentation, please let us know so that we can work with respective document owners to enhance the content accordingly.

Looking forward to your response.

Thank you
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-04-11T01:50:26.37+00:00

@Anonymous Just checking to see if you have any update on the above comment? Thanks
Meyliana Wangsa 5 Reputation points

2023-04-17T13:37:49.2033333+00:00
Hi @KranthiPakala-MSFT Thanks for your reply. I can run the SHIR windows container following the documentation (e.g. the SHIR container can connect and registered itself to the Az Synapse workspace, I can create file copy job from on-prem fileshare to Azure storage blob, etc). However the container keeps terminating every 30 minutes. When the container terminates, I see some errors on windows event log:

"An error was detected on device \Device\Harddiskx\DRxxxx during a paging operation."

"Disk x has the same disk identifiers as one or more disks connected to the system. Go to Microsoft's support website (http://support.microsoft.com) and search for KB2983588 to resolve the issue."

The container runs on a windows 2019 VM and I configured docker as per MSDN doc: https://learn.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?source=recommendations&tabs=dockerce Wondering if anyone else have had similar issue or know how to fix the issue
Darren-1990 21 Reputation points

2023-05-05T08:08:29.4133333+00:00

Same situation to original question - I am also looking to run an Azure Synapse SHIR in a Windows Container, however the container must not be hosted on a VM, but rather run as an Azure Container Instance - how to achieve this...?

1 answer

Your answer

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-04-06T01:13:41.3433333+00:00

Hi @Anonymous ,

Thanks for using Microsoft Q&A forum and posting your query.

Checking to see if you are facing any issue while following the steps provided in the documentation. If so, kindly elaborate a bit on what was the error message, and at which step you are noticing an issue so that we can assist you accordingly.

Or if you notice any steps that missing in the documentation, please let us know so that we can work with respective document owners to enhance the content accordingly.

Looking forward to your response.

Thank you
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-04-11T01:50:26.37+00:00

@Anonymous Just checking to see if you have any update on the above comment? Thanks
Meyliana Wangsa 5 Reputation points

2023-04-17T13:37:49.2033333+00:00

Hi @KranthiPakala-MSFT Thanks for your reply. I can run the SHIR windows container following the documentation (e.g. the SHIR container can connect and registered itself to the Az Synapse workspace, I can create file copy job from on-prem fileshare to Azure storage blob, etc). However the container keeps terminating every 30 minutes. When the container terminates, I see some errors on windows event log:

"An error was detected on device \Device\Harddiskx\DRxxxx during a paging operation."

"Disk x has the same disk identifiers as one or more disks connected to the system. Go to Microsoft's support website (http://support.microsoft.com) and search for KB2983588 to resolve the issue."

The container runs on a windows 2019 VM and I configured docker as per MSDN doc: https://learn.microsoft.com/en-us/virtualization/windowscontainers/quick-start/set-up-environment?source=recommendations&tabs=dockerce Wondering if anyone else have had similar issue or know how to fix the issue
Darren-1990 21 Reputation points

2023-05-05T08:08:29.4133333+00:00

Same situation to original question - I am also looking to run an Azure Synapse SHIR in a Windows Container, however the container must not be hosted on a VM, but rather run as an Azure Container Instance - how to achieve this...?

Answer 1

Hello Wangsa,
Running the Self-Hosted Integration Runtime (SHIR) in a Windows container is possible, and it can be a more efficient approach compared to running it on a Windows VM. The process involves creating a Docker container that hosts the SHIR service. Here's a general outline of the steps you need to follow:

Install Docker Desktop for Windows with Windows container support. If you're using a Windows Server, you can install the Docker Engine - Enterprise directly on the server.
Create a new directory for your Docker project, and within that directory, create a new file named Dockerfile with the following content:


# Use the Windows Server Core image as the base image

FROM mcr.microsoft.com/windows/servercore:ltsc2019

# Download and install the Self-Hosted Integration Runtime

ADD https://go.microsoft.com/fwlink/?linkid=2178493 "C:\shir\IntegrationRuntime.msi"

RUN msiexec /i "C:\shir\IntegrationRuntime.msi" /qn /norestart

# Set the working directory

WORKDIR C:/Program Files/Microsoft Integration Runtime/4.0/bin

# Expose the required ports

EXPOSE 8060 3389

# Run the SHIR service

CMD ["Microsoft.Cloud.DataMovement.DataTransferApp.exe"]

Open a command prompt or PowerShell window, navigate to the directory containing the Dockerfile, and run the following command to build the Docker image:


docker build -t shir:latest .

After the image is built, run a new container with the following command, replacing <NodeName> and <AuthKey> with the appropriate values from your Azure Data Factory or Synapse workspace:


docker run -d -p 8060:8060 -p 3389:3389 --name shir-container shir:latest -register -nodeName <NodeName> -authkey <AuthKey>

The SHIR container should now be running, and it will register with your Azure Data Factory or Synapse workspace.

Keep in mind that running SHIR in a Windows container is still subject to Windows licensing costs. If you want to reduce costs further, you can consider using the SHIR on a Linux container instead, following a similar approach.

For a more detailed explanation and a step-by-step guide, refer to the official documentation: How to run the self-hosted integration runtime in a Windows container

Share via

Running SHIR in Window Container

1 answer

Your answer