New columns not visible on Inspect Tab | Data Factory Pipeline

Question

New columns not visible on Inspect Tab | Data Factory Pipeline

Deniz Alkan 101

Hi everyone,

When I add new columns on Source component, they don't appear in the next component. Lets say there is 10 columns in Source Table and we add 2 new column and now we have 12 columns but in next component there is still 10 column in Inspect tab.

There is no Update or Import Schema options in Inspect tab. So I could not map new column to Sink.

Also I can see new columns in Data Preview tab. I can see data which this columns include but in Inspect tab there is no new columns so I can't use this new columns on Sink component

Is there any solution?

Thanks,
Deniz

Deniz Alkan 101 Reputation points

2022-09-06T11:25:23.147+00:00

UPDATE

When I refresh the page, I can see the columns which I just added. But it should not be solution :)
AnnuKumari-MSFT 26,716 Reputation points Microsoft Employee

2022-09-09T16:13:41.337+00:00

Hi @Deniz Alkan ,
Just checking in to see if you got a chance to see the last response on the thread. If it helped, please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well. If you have any further query do let us know.
AnnuKumari-MSFT 26,716 Reputation points Microsoft Employee

2022-09-13T09:28:06.663+00:00

Hi @Deniz Alkan ,
Just checking in to see if the below answer helped. Please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well. If you have any further query do let us know.
Deniz Alkan 101 Reputation points

2022-09-14T12:06:30.733+00:00

Hi @AnnuKumari-MSFT

I accepted thw answer. Thank you.

Deniz Alkan 101 Reputation points

2022-09-06T11:25:23.147+00:00

UPDATE

When I refresh the page, I can see the columns which I just added. But it should not be solution :)
AnnuKumari-MSFT 26,716 Reputation points Microsoft Employee

2022-09-09T16:13:41.337+00:00

Hi @Deniz Alkan ,
Just checking in to see if you got a chance to see the last response on the thread. If it helped, please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well. If you have any further query do let us know.
AnnuKumari-MSFT 26,716 Reputation points Microsoft Employee

2022-09-13T09:28:06.663+00:00

Hi @Deniz Alkan ,
Just checking in to see if the below answer helped. Please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well. If you have any further query do let us know.
Deniz Alkan 101 Reputation points

2022-09-14T12:06:30.733+00:00

Hi @AnnuKumari-MSFT

I accepted thw answer. Thank you.

Answer 1

AnnuKumari-MSFT 26,716 Microsoft Employee

Hi @Deniz Alkan ,
Welcome to Microsoft Q&A platform and thanks for posting your question here.
As I understand your issue, post the schema change in the source, you are not able to see the new schema in inspect tab of your mapping dataflow . Please let me know if that's not what your concern.

Kindly import schema in the source dataset or in the projection tab, try to import projection before you check the inspect tab.

Also, kindly make sure 'Allow schema drift' option is checked in the source transformation.

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Deniz Alkan 101 Reputation points

2022-09-07T13:06:48.51+00:00

Hi @AnnuKumari-MSFT

Thanks for your fast response.

But It's not about only Source component. For Example in Pıvot component I'm adding new columns. But in next component these columns does not appear
Deniz Alkan 101 Reputation points

2022-09-07T13:10:31.727+00:00

After I added these columns to Pivot component. I refreshed the page then columns appeared in Join component Inspect tab
AnnuKumari-MSFT 26,716 Reputation points Microsoft Employee

2022-09-07T16:32:08.667+00:00

Hi @Deniz Alkan ,
I tried to reproduce your scenario and observed the same series of event. I am looking into this with internal team if this is some kind of bug or an expected outcome. I will update once I hear back from them. Thankyou for your patience.
MarkKromer-MSFT 5,166 Reputation points Microsoft Employee

2022-09-08T00:26:30.703+00:00

When you add new columns from a Pivot, those columns are treated as "schema drift". You will see them in the data preview, but not in the Inspect because Inspect only knows about hardened metadata columns. To make those new columns part of your flow, click on the "Map drifted" button on the data preview tab of the Pivot transformation.
AnnuKumari-MSFT 26,716 Reputation points Microsoft Employee

2022-09-08T02:20:45.89+00:00

Thanks @MarkKromer-MSFT for your response.

@Deniz Alkan ,Kindly check the above solution and let us know if it resolves your issue.

----------

If the suggested response helped you, please click Accept Answer and kindly upvote the same. Thank you.
AnnuKumari-MSFT 26,716 Reputation points Microsoft Employee

2022-09-12T06:53:13.387+00:00

Hi @Deniz Alkan ,
Just checking in to see if the above answer helped. Please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well. If you have any further query do let us know.

New columns not visible on Inspect Tab | Data Factory Pipeline

0 additional answers

Activity