This error can occur in Windows 10 Fall Creators Update (version 1709) if you are using a shortcut to open Windows Defender, and that shortcut was created while using Windows 10 version 1703.
Windows Defender Antivirus had an updated experience that shipped with Windows 10 Creators Update (version 1703), which was designed to be easier to use and provide a comprehensive security experience. However, the old UI from version 1703 is no longer supported in version 1709.
To resolve the error, we recommend you delete any previous shortcuts you were using launch Windows Defender and create a new shortcut within Windows 10 version 1709.
If you are still having problems with Windows Defender, you can also use the Feedback Hub app as a great way to provide feedback directly to the product team. You can check out this article for steps to Send feedback to Microsoft with the Feedback Hub app.