How can I utilize multiple cores on Azure ML Studio VM's?

Question

Hi. I have a python script which can be run either sequentially or in parallel (using concurrent.futures). On my local machine using the parallel option results in a considerably faster execution (nearly linear speed up). Running the same script inside an Experiment on Azure ML Studio I was not able to observe any speedup from the parallel version. At first I thought adding the following line conda_env.docker.arguments = ["--cpuset-cpus=4"] would help, but still the same. Therfore my question is, how can I enable the docker container to leverage multiple cores of the vm-instance? Kind Regard Kai

Answer 1

Hello Kai,

You can easily change your VM with different cores according to your need. Please see following pic and let me know if you have any question.

Regards,
Yutong

Answer 2

Hello Yutonn,

I know how to change the #cores of my VM. The problem is, that these cores are not utilized.
I guess it's somehow related to this issue https://stackoverflow.com/questions/15639779/why-does-multiprocessing-use-only-a-single-core-after-i-import-numpy

Regards
Kai