This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 73%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
I am trying to ingest parquet data to snowflake. But I am facing issue when doing ingestion to snowflake due the ascii values present in between like shown below,
Ex : I have 2 columns like amount and amount_exceded like below
Column1
Amount - 10¿4564567¿¿
Column 2
Amount_exceeded - 24☐56756☐454
How to handle the above approach and ingest the data to snowflake.
P:S I am trying to ingest 450 tables
Thanks :)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 38%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EQ%3C/text%3E%3C/svg%3E)
Hello @Anandhakumar Cholendran,
Just checking if we are still connected on this discussion? Please let us know if you need to add more info so that we better assist you!
If the response is helpful, please click "Accept Answer" and upvote it. So that we can close this thread.
Hello @Anandhakumar Cholendran,
For the transformation, you can use the Derived Column transformation to replace special characters with either null or any appropriate substitute. You can use the replace() function to do this.
Hello @Anandhakumar Cholendran,
Just checking if we are still connected on this discussion? Please let us know if you need to add more info so that we better assist you!
If the response is helpful, please click "Accept Answer" and upvote it. So that we can close this thread.
Hello Sathish, Thanks for the thread. So I have few clarifications and need your help here :) Thanks in advance!!
Helo @Anandhakumar Cholendran
Yes, You are right, you can try the REGEXP_REPLACE function to replace all the special characters in a particular column and also make use of Lookup + ForEach in Dataflow to do the same for every column.
You have to be cautious if your input data contains special characters that are not supported by your output data format, your data flow may fail. Input data containing null values!!! input data volume is too large then it may fail the Data Flow!!!
Another way to handle this is to use any data profiling tool to analyze your input data (I mean pre-processing of data) and identify the special characters that are present in each column then you can use the REGEXP_REPLACE function to replace them in your data flow. This approach may be more time-consuming, but it can be more accurate and efficient in the long run.
In any case, it is important to thoroughly test your data flow and address any issues that arise to ensure the success of your data integration process.
If the response is helpful, please click "Accept Answer" it. So that we can close this thread.
Yes. Thanks for the clarification.