How to find the Azure Batch Python client library in Conda packaging

Question

How to find the Azure Batch Python client library in Conda packaging

Andrew Janke 25

I've started working with Azure Batch and use Python, with my Python environment managed by Anaconda. I'd like to install the Azure Batch Client Library and Azure Batch Management Client Library from the Azure SDK for Python in my Anaconda environments, preferably using conda instead of pip.

These are the azure.batch and azure.mgmt.batch modules (or "module packages"; whatever they're called), found in the azure-batch and azure-mgmt-batch PyPI packages. But I don't know where to find them in the Conda packaging of Azure SDK for Python.

The Azure SDK for Python introduced a Conda packaging of it back in 2021, and its use is described in the "How to install Azure library packages for Python" page. The Conda packaging differs from the PyPI packaging. The Python Azure SDK modules are packaged in to a smaller number of packages in the Conda form, with sometimes different naming.

Is the Azure Batch client library available in the Microsoft-supplied Conda packages somewhere (the ones in the microsoft conda channel, instead of the conda-forge channel)? If so, which Conda package?

And more generally, if I know what Azure SDK for Python module I want, or what PyPI package it's in, how can I find out which microsoft-channel Conda package it's in? I haven't been able to find a list of which module is in which Conda package anywhere.

There's an azure-batch conda package in the conda-forge channel (instead of the microsoft channel). But if I understand correctly, those conda-forge Azure packages are the old ones from before the 2021 introduction of the microsoft conda channel's packaging, and have different dependencies and stuff.

I'd prefer to install the Azure Batch client from the microsoft-channel Conda packages, instead of the conda-forge channel package or from PyPI/pip, for consistency with my other Azure Python packages, which are all installed from the microsoft-channel Conda packages. I've read that mixing interdependent packages from different channels can sometimes cause problems, and if you're mixing conda-managed and pip-managed packages in an Anaconda environment, you're supposed to install all the conda packages first, then the pip packages, and then don't go back and install or update any conda packages afterwards, or something like that.

Andrew Janke 25 Reputation points

2023-03-07T14:20:08.09+00:00

Thanks @Prrudram-MSFT !

That got me taken care of. I was able to get the Azure Batch client packages installed in our Conda environments using the conda-forge channel and pip/PyPI.

I'd like to mark your answer as Accepted here, but for some reason this page isn't letting me, even though I seem to be signed in with the same account that I posted my question with. (I only see thumbs-up flag icons under your answer; no "Accept" button.)

A couple notes for other people who come by here:

On the GitHub Issue I posted for this question, xiangyan99 from Microsoft chimed in with some more info:

https://anaconda.org/microsoft/repo only contains track 2 libraries. Unfortunately, azure-batch has not migrated to track 2 yet. Hence it is not on https://anaconda.org/microsoft/repo yet. And you are right. You need to use conda-forge or pypi to install them. To answer your question:
I'm guessing that means either its dependency versions close enough to what's in the microsoft packages that it didn't alter anything? Because it is still track 1, it depends on different libraries from other track 2 libraries. :)

So it seems that it's (probably) fine to use the conda-forge azure-batch package along with the other microsoft-channel packages. The azure-mgmt-batch package still needs to be grabbed from PyPI. I did conda install -c conda-forge azure-batch; pip install azure-mgmt-batch and it seems to have worked out so far.

NOTE for other users: There are issues with mixing conda and pip package installations in a Conda environment. You probably want to read that before doing so.

I also looked all through the documentation and couldn't find a readily available list of exactly which modules are in the microsoft-channel packages ("Track 2") either. So I guess you just have to go googling for each module if you have a question.
Andrew Janke 25 Reputation points

2023-03-07T14:21:40.8+00:00

Oh! @Prrudram-MSFT , it looks like you posted your response as a Comment and not an Answer, maybe? If you want to convert it to an Answer, I'll Accept it.
Prrudram-MSFT 13,371 Reputation points

2023-03-07T15:44:05.0666667+00:00

Hello @Andrew Janke

Thank you for kindly notifying me. I have converted it to answer. If you could accept it as an answer that would be great. :)

Andrew Janke 25 Reputation points

2023-03-07T14:20:08.09+00:00

Thanks @Prrudram-MSFT !

That got me taken care of. I was able to get the Azure Batch client packages installed in our Conda environments using the conda-forge channel and pip/PyPI.

I'd like to mark your answer as Accepted here, but for some reason this page isn't letting me, even though I seem to be signed in with the same account that I posted my question with. (I only see thumbs-up flag icons under your answer; no "Accept" button.)

A couple notes for other people who come by here:

On the GitHub Issue I posted for this question, xiangyan99 from Microsoft chimed in with some more info:

https://anaconda.org/microsoft/repo only contains track 2 libraries. Unfortunately, azure-batch has not migrated to track 2 yet. Hence it is not on https://anaconda.org/microsoft/repo yet. And you are right. You need to use conda-forge or pypi to install them. To answer your question:
I'm guessing that means either its dependency versions close enough to what's in the microsoft packages that it didn't alter anything? Because it is still track 1, it depends on different libraries from other track 2 libraries. :)

So it seems that it's (probably) fine to use the conda-forge azure-batch package along with the other microsoft-channel packages. The azure-mgmt-batch package still needs to be grabbed from PyPI. I did conda install -c conda-forge azure-batch; pip install azure-mgmt-batch and it seems to have worked out so far.

NOTE for other users: There are issues with mixing conda and pip package installations in a Conda environment. You probably want to read that before doing so.

I also looked all through the documentation and couldn't find a readily available list of exactly which modules are in the microsoft-channel packages ("Track 2") either. So I guess you just have to go googling for each module if you have a question.
Andrew Janke 25 Reputation points

2023-03-07T14:21:40.8+00:00

Oh! @Prrudram-MSFT , it looks like you posted your response as a Comment and not an Answer, maybe? If you want to convert it to an Answer, I'll Accept it.
Prrudram-MSFT 13,371 Reputation points

2023-03-07T15:44:05.0666667+00:00

Hello @Andrew Janke

Thank you for kindly notifying me. I have converted it to answer. If you could accept it as an answer that would be great. :)

Answer 1

Hi @Andrew Janke

The Azure Batch client library is not available in the Microsoft-supplied Conda packages. However, you can install the Azure Batch Client Library and Azure Batch Management Client Library from the PyPI packages azure-batch and azure-mgmt-batch using pip.

To install the Azure Batch Client Library and Azure Batch Management Client Library using pip, you can run the following command:

pip install azure-batch azure-mgmt-batch

Unfortunately, I couldn't find a list of which modules is in which Conda package. However, you can check the Microsoft Azure SDK for Python documentation for more information on how to install Azure library packages for Python.

https://learn.microsoft.com/en-us/azure/automation/python-packages
https://learn.microsoft.com/en-us/azure/machine-learning/v1/how-to-train-pytorch
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-environments

For any further help, you can raise a support ticket with azure batch technical support. If you don't have the ability to open a support ticket, you can send an email with your Subscription ID and link to this thread to AzCommunity@microsoft.com with subject line as "ATTN: Priyanka". I can then enable your subscription for one-time free support.

Hope this helps!

How to find the Azure Batch Python client library in Conda packaging

0 additional answers

Activity