HDBScan import error

Question

I trying to import hdbscan but it throws this following error

/databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py in import_patch(name, globals, locals, fromlist, level)
    156             # Import the desired module. If you’re seeing this while debugging a failed import,
    157             # look at preceding stack frames for relevant error information.
--> 158             original_result = python_builtin_import(name, globals, locals, fromlist, level)
    159 
    160             is_root_import = thread_local._nest_level == 1

hdbscan/_hdbscan_linkage.pyx in init hdbscan._hdbscan_linkage()

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject

When I update the numpy version to 1.22.0, I not getting this error but the problem is I'm using umap which tells the numpy version to be <1.20

To summarise
Databricks runtime version - 10.1 ML (includes Apache Spark 3.2.0, Scala 2.12)
Python Version - 3.8.10

Python packages installed

umap-learn==0.5.1
numpy==1.22.0
hdbscan==0.8.27
(This version throws error while importing umap)

Python packages installed

umap-learn==0.5.1
numpy==1.20.0
hdbscan==0.8.27
(This version throws the above error)

Any solution on this would be great.

Thanks

Answer 1

Hello @Rajamannar A K ,

Thanks for the question and using MS Q&A platform.

It's looks like compatibility issue with the hdbscan library.

You may checkout the Github issue which addressing the similar issue.

In case the above github issue doesn't solve your issue, I would request to open a new issue here: https://github.com/scikit-learn-contrib/hdbscan/issues

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators