I wanted to report that I have found the issue.
If the path to the file or the filename itself contains a pound sign # it looks like the previewer might be disregarding all the path that comes after the pound. Removing the # solves the issue.
If the # is the first character of a folder or a file, the previewer will show you the index of the folder.
If the # is in the middle of the file, the previewer will throw an error
I suspect like the previewer is not escaping the # character and therefore treats all characters after it as if it was a commentary, like when you write comments while programming.
I tested it with multiple machines and the same problems are happening on all of them. So far none of the updates I uninstalled seemed to have solved the problem.
This totally looks like a bug on the previewer.
I noticed and verified that this will happen on winver 20H2.
winver 20H1 does not have this issue.