Yes, you do, as clearly stated in the SPO service description document: https://learn.microsoft.com/en-us/office365/servicedescriptions/sharepoint-online-service-description/sharepoint-online-service-description
The fact that Microsoft might not directly enforce licensing requirements in code does not mean that you are allowed to use the product unlicensed, this is still a violation of the terms of service.