Gpu support in containers

Nick De Bruyckere 1

Hello

I have a container it's made from a .net core c# application together with imagemagick. OpenCL and clinfo are installed and I connected a gpu to it but when I run clinfo the number of platforms is 0. Why is this happening and how can I fix it do I need other things installed. i found https://github.com/NVIDIA/libnvidia-container. But the first packages you need to install are not being found on the container and when I try the drivers trough apt-get they're also not found. It might be since I don't have a base nvidia image. I'm trying this now. Maybe a list or steps of what I need (to do) to allow gpu support to my container, like what base image I should use to install asp.net on with my app, imagemagick and opencl.

Thank you for your time

singhh-msft 2,431 Reputation points

2021-03-16T06:41:06.313+00:00

@NickDeBruyckere, thank you for reaching out to us. Happy to help. There are 2 ways in which you can do this: Installing NVIDIA device plugin and Use the AKS specialized GPU image (preview). For both the methods, different images are used. Please do check out these links in detail and let me know if this helps.
Nick De Bruyckere 1 Reputation point

2021-03-16T08:53:49.7+00:00

i will thank you a lot already
singhh-msft 2,431 Reputation points

2021-03-16T08:56:30.213+00:00

Does the above comment answer your query completely? If yes, pls let me know, I will post it as an answer.
Nick De Bruyckere 1 Reputation point

2021-03-16T09:23:15.643+00:00

I don't think this really is what I'm looking for a solution so I can just create a container in my container registry add a GPU to it and that ImageMagick together with OpenCL is able to use this GPU. It is for testing purposes. I think this then has to do with my Docker file and the images I use no? I don't think the links you send are for Docker containers. I found that what I was testing was an unsupported version I'm now testing https://learn.microsoft.com/en-us/azure/container-instances/container-instances-gpu with the nvidia/cuda:9.0-runtime-ubuntu16.04 image as base. Woth my last test I had the problem that I couldn't access the cli do you know why this is? I also installed asp.net but my API also didn't respond.

Thank you in advance
singhh-msft 2,431 Reputation points

2021-03-16T13:10:26.873+00:00

I have shared links for AKS as you have added a tag of AKS, are you not using AKS? If yes, I can share different links with you. Further, I also came across this link, but did not share because I thought that you are using AKS. I assume you specifically want to use Nvidia GPU, that's why you are not using k8s-gcrio.azureedge.net/cuda-vector-add:v0.1 image from the documentation. Check out NVIDIA GPU Cloud Image For Microsoft Azure Release Notes and Using NGC with Azure Setup Guide and let me know if it helps.
Nick De Bruyckere 1 Reputation point

2021-03-16T17:42:03.06+00:00

The reason i'm not using that image is bacause i need to add my own api to it with imagemagick. I have a asp.net core 3.1 API I need to add and since the nvidia/cuda:9.0-runtime-ubuntu16.04 is supported i'm using this one and the site also says that image can be used as your base (I hope so at least the link only says the nvidia/cuda:9.0-base-ubuntu16.04 is supported but I asumed that the runtime would also be supported then). I'm currently trying to let my api run but when I build it in the cuda image I get an empty response and in the aspnet everything is fine so I have nu clue what I'm missing. I attached both files so you can see what I'm doing and hopefully what I'm missing. I also just tried it with the cuda-vector-add:v0.1 image but also an empty response. 78393-dockerfile-im-opencl.txt 78401-dockerfile-im.txt
Nick De Bruyckere 1 Reputation point

2021-03-17T10:18:57.133+00:00

i found the issue in the aspnetcore image are 2 env vars set DOTNET_RUNNING_IN_CONTAINER=true & ASPNETCORE_URLS=http://+:80/ and these were ofcoure not set in the nvidia image. Maybe you can add this to the install dotnet/aspnet core installs if installing for container add these env vars. Thank you for your time though.
singhh-msft 2,431 Reputation points

2021-03-17T11:33:32.11+00:00

Good to know that your issue is resolved. I will update this in the docs. I had already got in touch with the PG regarding this.

Further, is the whole setup working for you? Is there anything else I can help you with?
Nick De Bruyckere 1 Reputation point

2021-03-17T12:35:16.467+00:00

Well i hoped but what is actually working is the api i still have the problem with opencl that when I do clinfo that there are no platforms found even though there is a k80 connected to the container and opencl and OpenCL headers is installed. How can I make the gpu work with opencl since I want to test imagemagick with opencl with a gpu. How can I check if the gpu is accessible for the container? The nvidia-smi command is not found. I tried lshw -C display an this was the output.
78851-lshw.txt. This also adds a little to my problems... https://github.com/ImageMagick/ImageMagick/discussions/2545. But it's still weird that the gpu doesn't show up in the clinfo is there a fix for that?
singhh-msft 2,431 Reputation points

2021-03-18T09:56:58.39+00:00

Thanks for being patient, pls note that I am getting in contact with PG regarding this.
singhh-msft 2,431 Reputation points

2021-03-19T17:08:26.72+00:00

@Nick De Bruyckere , I confirmed from PG that you can go ahead and use Deploy GPU-enabled container instance - Azure Container Instances | Microsoft Learn for your ACI. It has information on base images to use and how to request quota for GPU SKUs.

Pls let me know if this answers your query.
singhh-msft 2,431 Reputation points

2021-03-22T03:56:58.993+00:00

@Nick De Bruyckere , just checking in to see if you got a chance to look at my response.
Nick De Bruyckere 1 Reputation point

2021-03-22T10:43:59.47+00:00

Yes this is the page i was using. But when I install opencl the and execute clinfo there are no platforms seen.
singhh-msft 2,431 Reputation points

2021-03-22T14:54:04.8+00:00

@Nick De Bruyckere , pls check answer by Praba here. Let me know if this helps.
Nick De Bruyckere 1 Reputation point

2021-03-23T10:21:28.597+00:00

i'll try it out thank you
Nick De Bruyckere 1 Reputation point

2021-03-23T12:10:27.293+00:00

i can't find how to look at the versions of the cuda drivers. I also try installing the latest version of smi but this gave an error after configuring the languages. But I couldn't look at the error since I need to continue with my internship and I need the cpu core space and time to continue testing since IM disables NVIDIA gpu's. So I'll leave this for now thank you for all the time and effort. When I need gpu's again in containers on Azure I'll get back to this oke? Sorry that I cannot finish this.
singhh-msft 2,431 Reputation points

2021-03-23T12:30:45.95+00:00

Please follow this to get Cuda version. And, if possible, let me know what error you are receiving, I will look into it.

Yes, sure please let me know if you need help on this further in future. Can we close this issue then?
Nick De Bruyckere 1 Reputation point

2021-03-29T13:58:22.767+00:00

Yes you can close this i'll reply to this again in the future if necessary thank you for your time.