I have some more clues about this. Turns out the black background only occurs when the files are in OneDrive folder. If I move them to a regular folder they look fine. The images are available offline in the OneDrive folder.
Similar issue occurs with font files too. In OneDrive they don't show any preview ('Abg' text doesn't show) but moving them to any other folder shows the preview.