Jothi-6650 avatar image
0 Votes"
Jothi-6650 asked HimanshuSinha-MSFT commented

Azure Databricks - Split column based on special characters in Databricks

I have a column in my csv file that possibly has value in below formats.

This has to be split as below respectively

When I try to split based on , i get error if second format arrives. So, i put an 'IF' condition as below.
Below is the python code I used.
if '
' in df['column'].values:
df[['column','newcolumn']] = df['column'].str.split('__', expand=True)
df['newcolumn'] = df['column']

code enters else condition even column has ''
tried below also - getting result as 'False' only
exists = '
' in df.column

found = df[df['column'].str.contains('__')]

Kindly help, how to handle this? any of you faced similar situation
Or, let me know if it will work if split by Q first, and then check for first alphabet (which will ignore initial numbers and __)

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @Jothi-6650 ,

I want to make sure that I really have an understanding of what you’re telling me.

You mentioned that " When I try to split based on , " , can you please let me know know where is they "," in the row which you mentioned ?

One of the desired output is mentioned "Value_-_10_counts",but the output input i think is "Q1_1_Value-10_counts" , Just wanted to check if the desired output should be "Value-_10_counts" ?

If you can share on row as you see in the .csv ( with dummy date ) that will be a big help /

Thanks Himanshu

0 Votes 0 ·

Hello @Jothi-6650,

We have not heard back from you on this . Can you please some test data as requested below .

Also if case if you have a resolution please do share that with the community as it will help other community members .

Thanks & stay safe


0 Votes 0 ·

0 Answers