Hi there,
Yes, Windows is included in the M365 E3 Unattended License. You can get more information from here new licensing options for Microsoft 365 for unattended scenarios
https://devblogs.microsoft.com/microsoft365dev/new-licensing-options-for-microsoft-365-for-unattended-scenarios/
If the reply is helpful, please Upvote and Accept it as an answer