Password policies in Active Directory can be defined only in the Default Domain Policy and it is applied at the root level of the domain.
That being said, it's possible to create another type of password policies and it's called "Fine-Grained Password Policy" or FGPP.
This policy can only be set on a user or a Domain Global Group.
AFAIK, it's not possible to apply a password policy on a OU.
Here is a good start on FGPP
https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/get-started/adac/introduction-to-active-directory-administrative-center-enhancements--level-100-#fine_grained_pswd_policy_mgmt
hth