Is it possible to create a pre-seeded docker image based on mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator?

Jef Schraag-Halma 21

To give our team a ready to query cosmos db container, pre-seeded with our data, I am trying to build a docker image based of mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator

My idea was to do something like this:

FROM mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest as cosmosdb

ENV AZURE_COSMOS_EMULATOR_PARTITION_COUNT=3
ENV AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1

RUN . /usr/local/bin/cosmos/start.sh &

# wait for the service to spin up
RUN sleep 20

# put code here to seed the db here by connecting to 127.0.0.1:8081

However, when I check the resulting container, it did not start any emulator process and my seeding code fails with a message that nothing is available at port 8081.

PRADEEPCHEEKATLA-MSFT 89,731 Reputation points Microsoft Employee

2023-03-31T05:56:16.7166667+00:00

@Jef Schraag Halma - We received your feedback that the answer provided on the thread was not helpful.

Kindly let us know what we could have done better to improve the answer and make your engagement experience good. We are here to help you and strive to make your experience better and greatly value your feedback.

Our engineer has provided a detailed answer which has clear steps for which you are looking for.

If you wish, you may re-surveying/rating for the engagement you received on the thread. Your feedback is very important to us.

Looking forward to you reply. Much appreciate your feedback!

Regards,

PRADEEPCHEEKATLA-MSFT

1 answer

Oury Ba-MSFT 19,331 Reputation points Microsoft Employee

2023-03-24T20:38:16.4133333+00:00
@Jef Schraag-Halma Thank you for reaching out.

It seems like you are trying to build a Docker image based on the Azure Cosmos DB Emulator for Linux. However, the Dockerfile you provided does not seem to start the emulator process correctly and failing with a message that nothing is available at port 8081.

Please try the below. I assume that you are using SQ API

docker run \ --publish 8081:8081 \ --publish 10250-10255:10250-10255 \ --memory 3g --cpus=2.0 \ --name=test-linux-emulator \ --env AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10 \ --env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true \ --env AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=$ipaddr \ --interactive \ --tty \ mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator

OR try increasing the sleep time in your Dockerfile to give the emulator more time to start up. For example, you can try changing the sleep 20command to sleep 30 or sleep 60 to see if that helps.

More troubleshooting tips:

Verify that the specific emulator container is in a running state.

Verify that no other applications are using emulator ports: 8081 and 10250-10255.

Verify that the container port 8081, is mapped correctly and accessible from an environment outside of the container.

Run the emulator on Docker for Linux (Preview)

Let me know if you are still having problems.

Regards,

Oury
Please sign in to rate this answer.
Oury Ba-MSFT 19,331 Reputation points Microsoft Employee

2023-03-27T05:47:22.3333333+00:00

@Jef Schraag Halma

Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Jef Schraag-Halma 21 Reputation points

2023-03-27T14:57:50.9633333+00:00

Hi Oury-Ba,

Your answer did not solve my problem. Incrementing the sleep value to 90 did not change the situation. Also note that the docker run example is not relevant because the issue occurs in the building the image, so even before I would want to run it.

Oury Ba-MSFT 19,331 Reputation points Microsoft Employee

2023-03-28T21:08:04.6133333+00:00

Jef Schraag Halma I am sorry to hear that the solution provided did not help.

Could you please change your first RUN command to:

RUN /bin/bash -c '/usr/local/bin/cosmos/start.sh &'

And report what happens?

Regards,

Oury

Oury Ba-MSFT 19,331 Reputation points Microsoft Employee

2023-03-29T16:52:11.32+00:00

Jef Schraag Halma

Were you able to check the above comment. Did you try the workaround I suggested?

Please let me know the result.

Regards,

Oury

Oury Ba-MSFT 19,331 Reputation points Microsoft Employee

2023-03-30T14:18:39.36+00:00

Jef Schraag Halma

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Jef Schraag-Halma 21 Reputation points

2023-04-20T13:22:49.19+00:00

@Oury Ba-MSFT , I believe I am one step closer. The flaw in my initial approach was to expect a process to continue to run when a Dockerfile RUN command completes. This it not the case.

A custom script script, to start the emulator and then run the seed script when the container is started is working ok. I used this line in my Dockerfile:

CMD ["/bin/bash", "-c","/usr/local/bin/cosmos/start-and-seed.sh"]

and this in my start-and-seed.sh:

#!/bin/bash # start the cosmos process export AZURE_COSMOS_EMULATOR_PARTITION_COUNT=5 export AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true export AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1 /usr/local/bin/cosmos/start.sh & echo "sleeping 30 secs" sleep 30 echo "starting seeder" # a seeder tool built in a previous stage /usr/local/bin/cosmos/MyCosmosEmulatorSeeder sleep infinity

When I run the image, I can navigate to the explorer and immediately see the seeded data.

What I also tried but what is not working yet is using data seeded during the docker build.

I had my docker file start-and-seed.sh (without the sleep infinity) as a RUN.

RUN /bin/bash -c './start-and-seed.sh' CMD ["/bin/sh", "-c", "#(nop) ", "CMD [\"/usr/local/bin/cosmos/start.sh\"]"]

From the docker build output, I saw that the data was inserted ok. However when the start.sh is run again for with my docker run, it clears the existing data.

I noticed that the start.sh script contains this fragment:

if [ -z "${AZURE_COSMOS_EMULATOR_CERTIFICATE}${AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE}" ]; then # Work around to remove old emulator data since restarting with existing data does not work at this time. if test -d "${COSMOS_APP_HOME}/appdata"; then rm -fr "${COSMOS_APP_HOME}/appdata" fi

are there any plans to get this fixed?

ps. Sorry for the late reply. Notifications were not working but that should be fixed now.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Is it possible to create a pre-seeded docker image based on mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator?

1 answer

Your answer