How can I set --ulimit memlock=-1

Daniel Otero Gómez 0 Reputation points
2024-03-05T22:04:30.24+00:00

I tried specifying it in the docker args but it didn't let me build the container. What I am trying to do is troubleshoot the shared memory of the VM like suggested [here]

How can I do this then?

I know there is a separate argument for the shm-size within the command function and I am working with it, but still, I would like to troubleshoot the memory limit as specified in the documentation.


# Create or update the component
    print("Creating job...")
    print(job_command)
    command_job = command(
        experiment_name="testing-ssl-byol",
        description=description,
        code=str(code_dir),
        environment=enviornment,
        inputs=inputs,
        outputs=outputs,
        command=job_command,
        compute="Testing-GPU-Cluster",
        distribution=MpiDistribution(process_count_per_instance=4),
        docker_args="--shm-size=1g --ulimit memlock=-1",
        environment_variables={"NCCL_DEBUG": "WARN"},#, "NCCL_IB_DISABLE": "0"},
        tags={"project": "ssl-research", "job-purpose": "testing"}
    )
    job = ml_client.jobs.create_or_update(command_job)
    print(f"Job created with ID: {job.id}")
Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
3,338 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.