Azure Databricks - Split column based on special characters in Databricks

Jothi 6 Reputation points

I have a column in my csv file that possibly has value in below formats.

This has to be split as below respectively

When I try to split based on , i get error if second format arrives. So, i put an 'IF' condition as below.
Below is the python code I used.
if '
' in df['column'].values:
df[['column','newcolumn']] = df['column'].str.split('__', expand=True)
df['newcolumn'] = df['column']

code enters else condition even column has ''
tried below also - getting result as 'False' only
exists = '
' in df.column

found = df[df['column'].str.contains('__')]

Kindly help, how to handle this? any of you faced similar situation
Or, let me know if it will work if split by Q first, and then check for first alphabet (which will ignore initial numbers and __)

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,214 questions
{count} votes