Combine columns from two sources

Question

Combine columns from two sources

Obaid UrRehman 171

Hi,

I have two sources resulting from some transformation in data flow:

I have tried using join, it replicates the data no matter join I select it outputs similar stuff:

I have tried union as well but union either creates null in columns (if done by name) or rows (if done by position)

SHouldnt the jojn just concat the columns together beacuse the IDs are same in both table.

Nasreen Akter 10,811 Reputation points Volunteer Moderator

2022-03-24T14:25:08.953+00:00

@Obaid UrRehman , what are the expected output you are looking for?
Obaid UrRehman 171 Reputation points

2022-03-24T14:32:34.453+00:00
Hi @Nasreen Akter

I want concat the version column to the first source so that it looks like this:

ID name value version 111 file1 0.1 3 111 file2 0.82 15 111 file3 2.2 2

Accepted answer

0 additional answers

Your answer

Nasreen Akter 10,811 Reputation points Volunteer Moderator

2022-03-24T14:25:08.953+00:00

@Obaid UrRehman , what are the expected output you are looking for?
Obaid UrRehman 171 Reputation points

2022-03-24T14:32:34.453+00:00

Hi @Nasreen Akter

I want concat the version column to the first source so that it looks like this:

ID name value version 111 file1 0.1 3 111 file2 0.82 15 111 file3 2.2 2

Answer 1

Nasreen Akter 10,811 Volunteer Moderator

Hi @Obaid UrRehman ,

Thank you for the ask. As you are joining by ID, that was the expected/correct output you were getting. Main problem here is, the Dataset2 with version does not have any indication which value should be considered for which ID+file. What you can do here:

Note: I have edited the comments: final output would be something like the foloowing:

for Dataset1, add a WINDOW

-for Dataset2, add SurrogateKey --> WINDOW

then do a JOIN with ID = ID and _ID1 = _ID2

Hope this helps, thanks! :)

Obaid UrRehman 171 Reputation points

2022-03-24T19:53:43.973+00:00
Hi @Nasreen Akter

Thank you so much for your answer.

I accept the answer. However, if possible, can you also share how to sort the data with fileName column (Assuming filename contains integers only).

The sort needs to be grouped, ie:

ID value version 111 1 x 111 3 z 111 12 y 112 1 a 112 3 b 112 12 c
Nasreen Akter 10,811 Reputation points Volunteer Moderator

2022-03-24T19:59:21.357+00:00

Hi @Obaid UrRehman , data already sorted by filename. Please see Window1 --> sort tab .
Nasreen Akter 10,811 Reputation points Volunteer Moderator

2022-03-24T20:02:53.613+00:00

@Obaid UrRehman , To sort you can use SORT in the Dataflow
Obaid UrRehman 171 Reputation points

2022-03-24T20:04:12.413+00:00

Hi,

my fileName is in string format, maybe thats why.! Ill try converting to int.

Thanks a lot :)
Nasreen Akter 10,811 Reputation points Volunteer Moderator

2022-03-24T20:06:03.287+00:00

@Obaid UrRehman , np, thanks! :)

Share via

Combine columns from two sources

0 additional answers

Your answer