It would certainly have helped to see the actual execution plans. And the code of the table-valued function, which I assume is an inline function?
I also like to remark that the queries looks unnecessarily complicated. I can't see any point with the JOIN. But I take it that this is a simplified version of your actual case.
Certainly, one would like to think that the queries are equivalent, and they would lead to the same internal representation, but apparently, this doesn't happen. It could be that, although the actual performance of the two plans are radically different, the estimated cost for the plans are closer, maybe so close that it is more or less a toss-up for the optimizer which plan to pick.
Keep in mind that the optimizer works with statistics about the data, which has been sampled, and from this information the optimizer makes an estimate of what is the best plan. This often works out well, so often that we spoiled and take it for granted. But there is more than one place when things can go wrong, and when they go wrong, they can go wrong with a bang and we see such drastic differences.