Which data type is equivalent to SQL Server Money to synpase delta lake table ?

Question

Which data type is equivalent to SQL Server Money to synpase delta lake table ?

Heta Desai 216

I am pulling table from SQL server where few columns are Money data type. default size of Money data type is (19,4). Target is Delta table. In Delta table I've used DECIMAL(19,4) data type. I am doing this using Synapse pipeline and dataflow. But It throws an error while writing to Delta table Decimal column.

Here is the error:

AnalysisException: Failed to merge fields 'rate' and 'rate'. Failed to merge decimal types with incompatible precision 19 and 10 & scale 4 and 0"

I am using metadata driven approach where there is MEtadata table in Azure sql server in which list of table to pull data from source is inserted. pipeline reads those metadata and loads data into target delta lake from source SQL server.

Answer 1

HimanshuSinha-msft 18,366 Microsoft Employee

Hello @Heta Desai ,
Thanks for the question and using MS Q&A platform.

As we understand the ask here is to get rid of the error , please do let us know if its not accurate.
I am assuming that on the creation of the delta tyable side you have selected the MERGE opertation , let me know if that correct ? I did tried to repro the issue as it worked fine for me , to me its due to the fact its realted to the date which is in the rate column . As you mentioned that you have a metadata table and I am assuming the you have the SELECT query or the column name against the table name . I suggest that you add a cast function in the metadata and try agian . It should work .

Something like .

select SomeMoney,CAST(SomeMoney as decimal(19,2)) from MoneyTypeTest

Please do let me if you have any queries.
Thanks
Himanshu

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
- If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

heta desai 1 Reputation point

2022-08-13T08:46:46.41+00:00

From SQL server i am reading the data and wiring to CSV file in adls. And using notebook i am reading that can from adls and writting it to delta table.

In meta data I have stored the select query ti read data from SQL server.
heta desai 1 Reputation point

2022-08-13T08:48:00.623+00:00

I also tried using dataflow where source will read data from SQL server and sink (inline database) writes to delta lake table. But it is also not working
heta desai 1 Reputation point

2022-08-13T08:55:54.283+00:00

For SQL server data types like decimal, numeric, money I have used decimal type with size same as SQL server. But at the time of data insertion it throws an error which I mentioned in question.
Heta Desai 216 Reputation points

2022-08-16T19:00:03.463+00:00
@HimanshuSinha-msft

I did try using CAST() in select query for Money column while pulling data from SQL Server table. Still it is giving same error. I save below SELECT wuery in metadata table which is used in dataflow to pull the data from source.

SELECT Id, CAST(rate AS decimal(19,4)) AS rate FROM dbo.testRate

> {
> "errorCode": "6002",
where the exception came from that shows a non-Pythonic\n 116 # JVM exception message.\n--> 117 raise converted from None\n 118 else:\n 119 raise\n\nAnalysisException: Failed to merge fields 'rate' and 'rate'. Failed to merge decimal types with incompatible precision 19 and 10 & scale 4 and 0",
> "failureType": "UserError",
> "target": "Write_ADLS_To_Staging",
> "details": []
> }
HimanshuSinha-msft 18,366 Reputation points Microsoft Employee

2022-08-17T19:28:40.65+00:00

Hello @Heta Desai ,
I did called this out in my reply also . Can you please check if the MERGE option is checked ?

, If yes what happens when you unselect it and run the pipeline ?

Thanks
Himanshu
HimanshuSinha-msft 18,366 Reputation points Microsoft Employee

2022-09-08T19:08:24.653+00:00

Hello @Heta Desai ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu\

Which data type is equivalent to SQL Server Money to synpase delta lake table ?

1 answer

Activity