@Asanka Kumarasingha Thanks for reaching out.
In case if you are using Azure SDK then the delay between each delivery failure and re-attempt is managed by the Service Bus SDK itself and is based on the retry policy that you have set. For more details you can refer to this section for service bus retry guidance.
Regarding your second question, it is not possible to remove an old message from the queue while it is in retry. Once a message is in the retry state, it will continue to be retried until it either reaches the maximum delivery count or is explicitly removed from the queue by the receiver. However, if a new message arrives in the queue while an old message is in retry, the new message will be processed normally and will not affect the retry status of the old message.
I hope this helps! Let me know if you have any other questions.
Please 'Accept Answer' if it helped so that it can help others in the community looking for help on similar topics.