question

ShambhuRai-4099 avatar image
0 Votes"
ShambhuRai-4099 asked KumarApparaju-6458 commented

The selected table is not delta table.



Hi Expert,

I am connecting Hive database using Azure Delta databricks datasets in azure data factory. connection and everything is successful but why i added in Sink and validate in copy data object .. it is giving validation error

197465-image.png


197502-image.png


Which dataset should use to load data into Hive table


azure-data-factoryazure-databricksazure-data-lake-storage
image.png (6.1 KiB)
image.png (21.9 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @ShambhuRai-4099

Just checking in to see if the below answer provided by @PratikSomaiya helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

1 Answer

PratikSomaiya avatar image
1 Vote"
PratikSomaiya answered KumarApparaju-6458 commented

Hi @ShambhuRai-4099

You can use Hive activity in ADF when working with Hive tables

Please refer to the below documentation:

https://docs.microsoft.com/en-us/azure/data-factory/connector-hive?tabs=data-factory

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@PratikSomaiya ,@PRADEEPCHEEKATLA-MSFT - facing same issue.

Using Hive activity will retain the delta functionality into the table? The table I have created in Azure databricks is a delta table . wondering why the Azure databricks delta lake linked service is not able to connect the table.

How to create a delta table that works with ADF >> Azure databricks Delta lake and Azure databricks metastore ?

0 Votes 0 ·
PratikSomaiya avatar image PratikSomaiya KumarApparaju-6458 ·

@KumarApparaju-6458

If you need to copy data from Hive datastore then it is recommended to use Hive activity

https://docs.microsoft.com/en-us/azure/data-factory/connector-hive?tabs=data-factory

Have you tried using Hive activity?

I understand that you created a Delta table, but if source is Hive then recommend to use Hive dataset as source dataset

0 Votes 0 ·