Share via

Data flow / Derived column doesn't see columns names - 'Column not found'

Andrzej 20 Reputation points
2025-02-10T17:03:05.9166667+00:00

Hello,

I am working with Derived Columns in Data Flow within Azure Synapse Analytics for the first time and am encountering some issues. Despite following tutorials and conducting simple tests, I receive an error message stating "Column not found". Also, I see in tutorials, that avaiable columns should be visibile in Dataflow Expresion builder, but not in my case. Both the source and sink are CSV files.

Could you please review the attached screenshots and advise on what corrections are needed?

Thank you, Andrzej

source settings:az1

columns view in Data preview:

az2

adding new colmn based on the string, not column - it's workingaz3

az4

error trying to add column based on the current column. First example:

az2

Another example + no columns visible in Dataflow Expresion builder:

az2b

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,380 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,655 questions
0 comments
Accepted answer
  1. phemanth 15,765 Reputation points Microsoft External Staff Moderator
    2025-02-10T18:56:43.68+00:00

    Hi @Andrzej

    Welcome to Microsoft Q&A platform and thanks for posting your query here.

    I have reproduced the scenario from my side with simple CSV file

    User's image

    I am able to add column without error

    User's image

    I am getting the columns in dynamic expression as wellUser's image Kindly check the below steps and confirm us if issue resolved.

    • Double-check the column mappings in your source settings. Ensure that the column names in your CSV file match exactly with what you are referencing in the Derived Column transformation.
    • If the columns are not visible in the Dataflow Expression Builder, try refreshing the schema or re-importing the source dataset. Sometimes, a refresh can help in recognizing the columns.
    • Ensure that the data types of the columns are correctly defined. Mismatched data types can sometimes cause issues with column recognition.

    I hope the above steps will resolve the issue, please do let us know if issue persists. Thank you

    1 person found this answer helpful.
    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Andrzej 20 Reputation points
    2025-02-10T23:11:46+00:00

    Hi @phemanth ,

    Thank you for your assistance in identifying that the root cause could be the source.

    Upon reproducing a simple CSV file, I found it to be functional. This led me to investigate the issues with my CSV files generated from SAP data imports. I concluded that during the integration process, the dataset should have been marked as 'Import schema/From connection/store'. Without this, my files had an empty schema, even though the Preview recognized the columns.

    Thank you, Andrzej

    azz1

    aaz3

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.