BUG : Flatten transformation not reading the array column with multiple words in column name

Question

I was trying to split the pipe delimited string column which had a column name {Bill Code} in Derived column transformation followed by Flatten transformation, Derived transformation which will split the string and convert it into an array and Flatten will convert them into individual rows.
Ex: B12|B13|B14 to ["B12", "B13", "B14"]

Then I passed this column with {Bill Code} name to flatten transformation and went to data preview it threw an error "Unroll value {Bill Code} must be an array", although the bill code column was an array but it was not honoring it, then I had to change the column name from bill code to code and then the flatten transformation honored the column as an array.

Conclusion : Flatten transformation is not honoring the array column with multiple words separated by space and enclosed by curly brackets .

Answer 1

I have reproduced this bug using data:

Row,Bill Code
1,A1|A2|A3
2,B12|B13|B14

and script:

source(output(
        Row as short,
        {Bill Code} as string
    ),
    allowSchemaDrift: true,
    validateSchema: false,
    ignoreNoFilesFound: false,
    format: 'delimited',
    container: 'martin',
    fileName: 'sample.csv',
    columnDelimiter: ',',
    escapeChar: '\\',
    quoteChar: '\"',
    columnNamesAsHeader: true) ~> origin
origgin derive(Code = split({Bill Code},'|')) ~> Split
Split foldDown(unroll(Code),
    mapColumn(
        Row,
        Bill = Code
    ),
    skipDuplicateMapInputs: false,
    skipDuplicateMapOutputs: false) ~> Flatten1

I tried several permutations of names and flatten settings, in case something else was going on. However it looks like you are right, it is the name. This seems odd to me. I will let the developers know about this bug. Thank you for bringing it to our attention.

Answer 2

This bug should now be fixed.