Functions like StrCmpW and VarBstrCmp do a lot more work internally to deal with locales and sorting than the wcscmp CRT function. For example, after getting sorting information StrCmpW internally calls kernel32.dll!SortCompareString(void).
The documentation at Handling Sorting in Your Applications states "The functions CompareString, CompareStringEx, lstrcmp, lstrcmpi, LCMapString, LCMapStringEx, FindNLSString, and FindNLSStringEx all default to use of a "word sort" technique. For this type of sort, all punctuation marks and other nonalphanumeric characters, except for the hyphen and the apostrophe, come before any alphanumeric character." The '~' character is nonalphanumeric and in my opinion that is the reason for the difference in the comparison results.