Yes, your understanding is correct. If the activity fails, the retry attempt happens. If pipeline is successful, this retry setting is ignored in success case
Reference - https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview#resume-from-last-failed-run
Activity level retry: You can set retry count on copy activity. During the pipeline execution, if this copy activity run fails, the next automatic retry will start
----------
Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav