can i Connect to Azure Hdinsight Hive DB using Python script???

Vikrant Vikrant 0

Hello everyone!!!

I am having a task to perform on Azure Hdinsight Hive DB and Azure databrick.

I have to connect to Azure Hdinsight Hive DB and get all data in excel or csv formate on daily bases automatical that will store in my Storage account (blob storage).

i have tried to connect to Azure Hdinsight Hive DB using a python script in azure data brick and automate it to perform the task .

but i am unable to connect to Azure Hdinsight Hive DB using python script.

it showes error as below

Could not connect to any of [('20.240.139.132', 10000)]

so if someone have python script to connect to Azure Hdinsight Hive DB please let me know.

i have tried it with pydbo and pyhive.

Else any other way to do same task.

Thanks

Vikrant

PRADEEPCHEEKATLA-MSFT 85,026 Reputation points Microsoft Employee

2023-03-08T05:44:32.0266667+00:00

Hello @Vikrant Vikrant ,

Thanks for the question and using MS Q&A platform.

Could you please share the code snippet which you are executing along with the screenshot or stacktrace of the error message which you are experiencing?
Vikrant Vikrant 0 Reputation points

2023-03-09T09:33:42.8966667+00:00

Vikrant Vikrant 0


# driver = 'com.microsoft.sqlserver.jdbc.SQLServerDriver'
# conn_string = f"DRIVER={driver};SERVER={url}"

# connect = f'DRIVER={driver};SERVER={host};DATABASE={database};UID={username};PWD={password}'
    # logging.info(connect)
# conns = pyodbc.connect(connect)
# # cnxn = pyodbc.connect('DSN=spark')


conn = hive.connect(host=host, port=10000, username=username, password=password, database=database, auth='LDAP', configuration={'ssl': 'true', 'transportMode': 'http'})

# # Create the PyHive connection object
# # connection = hive.Connection(host=host, port=port, username=username, password=password, auth='CUSTOM')
# # conn = hive.connect(host=host,
# #                     port=port,
# #                     database=database,
# #                     auth='LDAP',
# #                     username=username,
# #                     password=password)
print("connected")

Response:- Could not connect to any of [('20.240.139.132', 10000)]

i have tried to connect using both python Pyhive and Pyodbc but unable to connect.

PRADEEPCHEEKATLA-MSFT 85,026 Reputation points Microsoft Employee

2023-03-09T09:38:13.3333333+00:00

@Vikrant Vikrant Can you please copy and share the complete stack trace here?
Vikrant Vikrant 0 Reputation points

2023-03-09T11:54:51.9+00:00

First we have created a HDinsight Hadoop Cluster.

Then we created a table in Hive DB and import our big data.

then we have try to connect to HDinsight using Databrick notebook.

We also tried to execute script using ADF in hive but that also failed.

Hear are result for ADF script execution on hive DB.
PRADEEPCHEEKATLA-MSFT 85,026 Reputation points Microsoft Employee

2023-03-10T04:20:16.39+00:00
@Vikrant Vikrant Thanks for sharing the details.

From the error message it looks like user configuration issue - Error 500, Internal Server Error can occur due to many reasons. Here are a few error sources.

Error occurs due to intermittent/transient issues.

Error occurred in the configuration of the webserver.

Error occurs due to permissions issues at client-server.

Error occurs due to time out.

If the issue is not resolved yet or you still experience the issue consistently, I would recommend raising a support ticket for the support engineer to investigate.
PRADEEPCHEEKATLA-MSFT 85,026 Reputation points Microsoft Employee

2023-03-13T05:30:06.67+00:00

@Vikrant Vikrant

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Share via

can i Connect to Azure Hdinsight Hive DB using Python script???