How to increase default compute instance for azure kubernetes compute clusters while creating it using python SDK??

Question

How to increase default compute instance for azure kubernetes compute clusters while creating it using python SDK??

Aryan Rastogi 60

First I created my cluster using below code:

attach_config = KubernetesCompute(
                            name=kubernetes_attached_compute,
                            resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}",
                            properties={
                                "ssl": {
                                    "enabled": True
                                }
							}
				)

got default instance of

User's image

Then i created using the below code:

attach_config = KubernetesCompute(
                            name=kubernetes_attached_compute,
                            resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}",
                            properties={
                                "ssl": {
                                    "enabled": True
                                },
                                "clusterPurpose": cluster_purpose,
                                "defaultInstanceType": "custominstancetype",
                                "instanceTypes": {
                                    "custominstancetype": {
                                        "resources": {
                                            "limits": {
                                                "cpu": "4",
                                                "memory": "6Gi"
                                            },
                                            "requests": {
                                                "cpu": "0.5",
                                                "memory": "1Gi"
                                            }
                                        }
                                    }
                                }
                            }
                        )

But this creates 2 instances User's image

I only need one with limit 4 cores and 6Gi

How to do that??

Nikhil Duserla 7,540 Microsoft External Staff Moderator

Hello @Aryan Rastogi,

Are you looking to create a custom instance with a limit of 4 cores and 6 GiB of memory only? If so, please try using the following script:

attach_config = KubernetesCompute(
    name=kubernetes_attached_compute,
    resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}",
    properties={
        "ssl": {
            "enabled": True
        },
        "resources": {
            "limits": {
                "cpu": "4",
                "memory": "6Gi"
            }
        }
    }
)

If you have any further queries, do let us know.

Aryan Rastogi 60 Reputation points

2025-05-30T04:40:22.75+00:00

even after using code above default is created with this config not 4core and 6gi

Arko 3,490 Microsoft External Staff Moderator

I think the schema must include both defaultInstanceType and the matching entry inside instanceTypes. Can you try this once?

from azure.ai.ml.entities import KubernetesCompute
attach_config = KubernetesCompute(
    name=kubernetes_attached_compute,
    resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}",
    properties={
        "ssl": {
            "enabled": True
        },
        "defaultInstanceType": "custominstancetype",
        "instanceTypes": {
            "custominstancetype": {
                "resources": {
                    "limits": {
                        "cpu": "4",
                        "memory": "6Gi"
                    },
                    "requests": {
                        "cpu": "0.5",
                        "memory": "1Gi"
                    }
                }
            }
        }
    }
)

Let me know how it goes, we can then troubleshoot accordingly.

Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-02T04:46:59.2233333+00:00

Following up on this Aryan Rastogi. Did you try out the suggestion? Let me know how it went. Thanks
Aryan Rastogi 60 Reputation points

2025-06-02T09:01:26.0066667+00:00

yeah I tried that earlier it creates 2 instances as i mentioned in my original question
Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-03T06:45:35.47+00:00

Okay, I found a document from MS it says- https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-kubernetes-instance-types?view=azureml-api-2&tabs=select-instancetype-to-trainingjob-with-cli%2Cselect-instancetype-to-modeldeployment-with-cli%2Cdefine-resource-to-modeldeployment-with-cli#create-a-custom-instance-type

can you share the link of the repo from where you are testing this code or if possible share the complete code which you are trying. Edit your original question to add that. It will help me to understand the issue better and narrow down on it. Thanks
Aryan Rastogi 60 Reputation points

2025-06-04T21:00:35.9166667+00:00

I have shared the code i used above. What else is required i didn't understand what you were asking for ??
Aryan Rastogi 60 Reputation points

2025-06-04T21:01:29.4+00:00

In the link you gave they are using a YAML file for custom computer creation I require python sdk
Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-05T05:07:04+00:00
Hello Aryan Rastogi, can you once edit this section once. I made a small change to the above code which I shared earlier.

properties={ "ssl": {"enabled": True}, "instanceTypes": { "custominstancetype": { "resources": { ... }, "description": "Only custom instance" } }, "defaultInstanceType": "custominstancetype" }

Here, instanceTypes is declared before defaultInstanceType, kindly define instanceTypes first, then defaultInstanceType after that kindly validate the attached compute using

compute = ml_client.compute.get(kubernetes_attached_compute) print(compute.properties.get("instanceTypes"))

Please try this once and let me know, meanwhile I will try to get some inputs from the MS internal ML product group and check on the feasibility of this requirement. Thanks

Aryan Rastogi 60

Still getting 2 compute
User's image

Code used:

attach_config = KubernetesCompute(
            name=kubernetes_attached_compute,
            resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}",
            properties={
                "ssl": {"enabled": True},
                "clusterPurpose": cluster_purpose,
                "instanceTypes": {
                    "custominstancetype": {
                        "resources": {
                            "limits": {"cpu": "4", "memory": "6Gi"},
                            "requests": {"cpu": "0.5", "memory": "1Gi"},
                        }
                    }
                },
                "defaultInstanceType": "custominstancetype",
            },
        )

Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-06T10:47:46.02+00:00

Hello Aryan Rastogi, please check your private message. I have requested few details from you.

Your answer

Nikhil Duserla 7,540 Reputation points Microsoft External Staff Moderator

2025-05-29T13:53:54.3366667+00:00

Hello @Aryan Rastogi,

Are you looking to create a custom instance with a limit of 4 cores and 6 GiB of memory only? If so, please try using the following script:

attach_config = KubernetesCompute( name=kubernetes_attached_compute, resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}", properties={ "ssl": { "enabled": True }, "resources": { "limits": { "cpu": "4", "memory": "6Gi" } } } )

If you have any further queries, do let us know.
Aryan Rastogi 60 Reputation points

2025-05-30T04:40:22.75+00:00

even after using code above default is created with this config not 4core and 6gi
Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-05-30T13:19:35+00:00

I think the schema must include both defaultInstanceType and the matching entry inside instanceTypes. Can you try this once?

from azure.ai.ml.entities import KubernetesCompute attach_config = KubernetesCompute( name=kubernetes_attached_compute, resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}", properties={ "ssl": { "enabled": True }, "defaultInstanceType": "custominstancetype", "instanceTypes": { "custominstancetype": { "resources": { "limits": { "cpu": "4", "memory": "6Gi" }, "requests": { "cpu": "0.5", "memory": "1Gi" } } } } } )

Let me know how it goes, we can then troubleshoot accordingly.
Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-02T04:46:59.2233333+00:00

Following up on this Aryan Rastogi. Did you try out the suggestion? Let me know how it went. Thanks
Aryan Rastogi 60 Reputation points

2025-06-02T09:01:26.0066667+00:00

yeah I tried that earlier it creates 2 instances as i mentioned in my original question
Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-03T06:45:35.47+00:00

Okay, I found a document from MS it says- https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-kubernetes-instance-types?view=azureml-api-2&tabs=select-instancetype-to-trainingjob-with-cli%2Cselect-instancetype-to-modeldeployment-with-cli%2Cdefine-resource-to-modeldeployment-with-cli#create-a-custom-instance-type

can you share the link of the repo from where you are testing this code or if possible share the complete code which you are trying. Edit your original question to add that. It will help me to understand the issue better and narrow down on it. Thanks
Aryan Rastogi 60 Reputation points

2025-06-04T21:00:35.9166667+00:00

I have shared the code i used above. What else is required i didn't understand what you were asking for ??
Aryan Rastogi 60 Reputation points

2025-06-04T21:01:29.4+00:00

In the link you gave they are using a YAML file for custom computer creation I require python sdk
Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-05T05:07:04+00:00

Hello Aryan Rastogi, can you once edit this section once. I made a small change to the above code which I shared earlier.

properties={ "ssl": {"enabled": True}, "instanceTypes": { "custominstancetype": { "resources": { ... }, "description": "Only custom instance" } }, "defaultInstanceType": "custominstancetype" }

Here, instanceTypes is declared before defaultInstanceType, kindly define instanceTypes first, then defaultInstanceType after that kindly validate the attached compute using

compute = ml_client.compute.get(kubernetes_attached_compute) print(compute.properties.get("instanceTypes"))

Please try this once and let me know, meanwhile I will try to get some inputs from the MS internal ML product group and check on the feasibility of this requirement. Thanks
Aryan Rastogi 60 Reputation points

2025-06-05T07:36:48.6466667+00:00

Still getting 2 compute

Code used:

attach_config = KubernetesCompute( name=kubernetes_attached_compute, resource_id=f"/subscriptions/{config_dict['subscription_id']}/resourceGroups/{resource_group}/providers/Microsoft.ContainerService/managedClusters/{kubernetes_name}", properties={ "ssl": {"enabled": True}, "clusterPurpose": cluster_purpose, "instanceTypes": { "custominstancetype": { "resources": { "limits": {"cpu": "4", "memory": "6Gi"}, "requests": {"cpu": "0.5", "memory": "1Gi"}, } } }, "defaultInstanceType": "custominstancetype", }, )
Arko 3,490 Reputation points Microsoft External Staff Moderator

2025-06-06T10:47:46.02+00:00

Hello Aryan Rastogi, please check your private message. I have requested few details from you.

Share via

How to increase default compute instance for azure kubernetes compute clusters while creating it using python SDK??

Your answer