Join error

arkiboys 9,686 Reputation points

Do you see why I get the error in the first Join which is Join1?
'Join1': Non-equality joins should have broadcasted one of the join sides

Thank you


Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,131 questions
0 comments No comments
{count} votes

Accepted answer
  1. MarkKromer-MSFT 5,206 Reputation points Microsoft Employee

    Non-equi joins require broadcast join in Spark in order to resolve the non-equality. Go into the Optimize tab on your join and select Broadcast Fixed, then choose the Fixed broadcast option.

2 additional answers

Sort by: Most helpful
  1. Ali Asgar 11 Reputation points Microsoft Employee

    For anyone who stumbles upon here trying to find an answer:

    The underlying problem (and thus the real question) is how to correctly use dynamic columns in lookup or join activities in Data Flow transformations. The following screenshot answers this question.


    You need to use expr and concat functions to correctly use parameters. The 'Source@" and 'Destination@' strings refer to the names of the Data Flow activities that feed into the Join/Lookup.

    2 people found this answer helpful.
    0 comments No comments

  2. Mark Kromer MSFT 1,146 Reputation points

    Does the error clear when you set the broadcasting on?