If an attribute is mapped from source -> target, Entra Provisioning will look at the target system (SCIM) object and see if that attribute is present and if it is populated with the right value. If the right value isn't present, the provisioning service will try to populate it. On any subsequent sync cycle that evaluates that object, the same check will happen and if the value isn't present, again it will be added.
If your service doesn't support displayName, it should not be mapped from Entra -> SCIM app. Removing that mapping will eliminate the inclusion of the attribute and any repeated attempts to "fix" the missing data issue.