Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions
A refresh did the trick. solved.
Thanks anyway
copy activity sink error - too many columns
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 97%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
arkiboys
9,686
Reputation points
Hello,
Copy activity sink is the parquet file
It appears if there are a-lot of varchar(max) columns in source, the copy sink can not handle it.
Is there a limitation to the number of columns can be used in sink of the copy activity to populate parquet file?
Thanks
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT 46,442 Reputation points Microsoft Employee2021-12-01T22:36:22.033+00:00 Hi @arkiboys ,
Thanks for your query. Could you please share the complete error message along with failed pipeline runID ? Also could you please tell us how many columns do you have in source data and sink? Also please confirm if you are doing auto-mapping or explicit mapping?
Thank you
Sign in to comment
Accepted answer
-
arkiboys 9,686 Reputation points
2021-12-02T06:15:56.543+00:00 -
KranthiPakala-MSFT 46,442 Reputation points Microsoft Employee
2021-12-02T19:06:49.8+00:00 Hi @arkiboys ,
Glad to know you issue was resolved. Thanks for confirming. Please feel free to open a new thread if the issue resurfaces. Also please feel free accept your own answer as it could help other community members.
Thank you.
Sign in to comment -