I don't see why you're encountering that error unless there are no enabled users in the organizational unit.
Run this to verify that you get the expected results:
$c = @"
EmailAddress
******@z.com
******@z.com
******@z.com
"@
$csv = $c | ConvertFrom-Csv
$a = @"
Name,EmailAddress
Adam,******@z.com
Beth,******@z.com
Carol,******@z.com
Doug,******@z.com
"@
$ad = $a | ConvertFrom-Csv
Compare-Object -ReferenceObject $csv.EmailAddress -DifferenceObject $ad.EmailAddress -IncludeEqual -ExcludeDifferent |
Select-Object -Property InputObject |
Export-Csv -Path C:\junk\match.csv -NoTypeInformation
Compare-Object -ReferenceObject $csv.EmailAddress -DifferenceObject $ad.EmailAddress |
Select-Object -Property InputObject |
Export-Csv -Path C:\junk\notmatchusers.csv -NoTypeInformation