Combine two columns into one Azure Data Factory - Data Flow

Question

Hello

I have the following problem. I'm doing a data flow in ADF and the first thing I do is to decouple a JSON file via its body. When I decouple it I get two columns called "humidity" and "Hum" which work as follows. In some rows the humidity column contains the humidity data and in that row Hum is NULL and in others it is the other way around. How can I combine both columns into one so that I can delete both and the resulting column has the humidity data? The value is of type float.

Thank you

Answer 1

Hello @Andres Esteban ,
Thanks for the question and using MS Q&A platform.

As I understand, you want to combine two columns into one. One of the columns is sometimes null.

If your data is like:

Humidity | Hum | desired result  
23.4 | 23.4 | 23.4  
55.1 | Null | 55.1  
Null | 12.2 | 12.2  

Then you can use coalesce( Humidity , Hum) in derived column. Coalesce takes the first non-null value. This works logically because you never have 2 numbers in disagreement. However if you have data like:

Humidity | Hum  
12.3 | Null  
45.2 | 1.9  
63.1 | 8.1  

Things become much more complicated. What is the desired output when 2 non-null values disagree? Is it an average value, or a concatenation or something else? In such a case, coalesce would always take one column and ignore the other.

Please do let me if you have any queries.

Thanks
Martin

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Hi @MartinJaffer-MSFT ,
what would be your suggestion, if all column values were needed, as you described, that coalesce would always drop the second value?