Azure Dataflows -Unable to source from lake databases

Pradeep Peri 1

Facing an issue when i try to source from Lake data base - below is the error :

at Source 'source1': org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:null);

I am able to query the lake databases using sql scripts but failing only when i am trying to use them as a source in dataflows.

KranthiPakala-MSFT 46,492 Reputation points Microsoft Employee

2022-08-31T00:13:50.29+00:00

Hello @Pradeep Peri ,

Thanks for the question and using MS Q&A platform.

I tried the same and was able to reproduce the same error you are seeing.
I'm escalating this to respective product owners and will get back to you as soon as we have an update on the same.

Thank you for your patience.

Hi @MarkKromer-MSFT - Could you please review this issue in Datalflow - Lake database (workspaceDB) issue and share your thoughts?
Nedda Marhoon 6 Reputation points

2022-09-01T00:06:49.087+00:00

I have the same issue started 2 days ago.. please help.
Wilson, Chris 6 Reputation points

2022-09-01T15:06:24.643+00:00

We've been seeing the same thing with regards to both source projection and sink mapping for lake databases since 8/26.

Support request ID: 2208300040004804
Pradeep Peri Venkata 1 Reputation point

2022-09-01T21:03:24.557+00:00

Hey Kranthi @KranthiPakala-MSFT any updates for me ? i also raised a support ticket for this - 2208300030000411
Pradeep Peri Venkata 1 Reputation point

2022-09-01T21:05:17.01+00:00

Hey Kranthi @KranthiPakala-MSFT any updates for me ? i also raised a support ticket for this - 2208300030000411
BobGang-3117 6 Reputation points

2022-09-02T12:39:43.48+00:00

Can we get an update, please? This is impacting all Azure Synapse Workspace Data Flows.
KranthiPakala-MSFT 46,492 Reputation points Microsoft Employee

2022-09-02T23:19:25.63+00:00

Hi @Pradeep Peri , @BobGang-3117 , @Nedda Marhoon , @Wilson, Chris .

Sorry for your experience. I have escalated this issue directly to the respective product owners and awaiting response. Will keep you posted as soon as we have an update.

Thank you

@MarkKromer-MSFT , @Sunil Sabat - FYI
Nedda Marhoon 6 Reputation points

2022-09-08T02:34:02.957+00:00

Here is the latest update from Microsoft that I got through the support ticket that I raised a couple days ago:

As per the latest update from product group, they have a hotfix but it has not yet been deployed. I will send another follow up to them today and request them to deploy the fix ASAP.
Wilson, Chris 6 Reputation points

2022-09-09T13:51:29.027+00:00

Have you heard anything further from your ticket? The representative from my ticket said that it was a known defect and indicated we could switch our sources and sinks to inline or datasets as a work-around, but that would involve a large amount of re-work here, so we'd rather wait for a proper fix, but I've yet to hear a timeline on the actual patch.
Josue Preciado Preciado 1 Reputation point

2022-09-18T14:31:13.553+00:00

This is happening to us as well. Did your rep do anything in particular to solve for this issue?
KranthiPakala-MSFT 46,492 Reputation points Microsoft Employee

2022-09-30T16:43:39.383+00:00

Hi @Anonymous ,

Could you please elaborate a bit on your problem along with few screen shots of the problem?

The product team had rolled out a hot fix to all regions on 9/8/2022 and we have also verified that the issue has been resolved.

Also, you can see confirmation from @Wilson, Chris , @BobGang-3117 below. But in case if you are still experiencing the issue, please do share few screenshots and also share your ADF, Azure IR region details which would help us to replicate the issue on our end.

We look forward to your confirmation.

Thank you

1 answer

Wilson, Chris 6 Reputation points

2022-09-09T16:30:03.123+00:00

We just heard back from our support rep indicating the issue has been patched, and I confirmed I can now successfully import schema from sources that are Lake DB Tables as well as import sink mappings, so if you were having the same issue, check to see if it's resolved for you as well.
Please sign in to rate this answer.

1 person found this answer helpful.
BobGang-3117 6 Reputation points

2022-09-09T16:45:48.043+00:00

We have a winner, I am able to use Data Preview in dataflows. Thank you.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Azure Dataflows -Unable to source from lake databases

1 answer

Your answer