ADF copy data from parquet file to snowflakes, if column title has special char (#), the daynamic mapping not works

Dong, Jacky 21

I have a parquet file ADF to Snowflakes. One column title of the parquet file has special char (#).

When import mapping, I can manually add column mapping and the ADF result is ok.

But I try to use dynamic mapping, the ADF result, all columns except CCATO# is correct. CCATO# is NULL.

May I know how to use dynamic copy mapping for column has special char?

Dong, Jacky 21 Reputation points

2022-08-26T07:31:10.227+00:00

I try to use "ordinal" instead of source name, still not work.

Here is the ADF result. only CCATO column is NULL, all other columns are correct.
MartinJaffer-MSFT 25,451 Reputation points Microsoft Employee

2022-08-26T22:08:58.433+00:00

Wow.

With your earlier example of manual vs dynamic mapping I thought it might be some weird problem in Data Factory, but now that you show me ordinals only not working, I must reconsider my view. I can't imagine any reason the ordinal would fail. Just to be sure, are ANY rows expected to have CCATO null?

Hmm. Parquet can be wierd at times. I don't have a snowflake set up to repro with. One thing to test, if you change sink from Snowflake to delimited text, does CCATO still come out null?
Dong, Jacky 21 Reputation points

2022-08-29T02:24:18.05+00:00
Thank you for your reply.

No other rows null except CCATO

I have tryied sink as csv file. looks ok. but weird is the column title always CCATO#, even I force change to CCATO
Dong, Jacky 21 Reputation points

2022-08-29T02:29:07.837+00:00

continue last post.

Sink to csv looks ok, but my destination is Snowflake.

Strange thing is if I use customized mapping instead of dynamic mapping, the result is ok. looks lik it's dynamic copy mapping JSON translator issue.

Activity