Hello VarunArora,
Please check the solution in the next official article: https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/credentials-prompt-access-webdav-fqdn-sites
Then set OpenDocumentsReadWriteWhileBrowsing DWORD Value on a client machine under the key:
HKEY_CURRENT_USER\Software\Microsoft\Office\<version>\Common\Internet.
Set the Value data to 1.
Additionally, you can considering enabling Cookie Authentication to ease the way of access from your office applications:
https://learn.microsoft.com/en-us/office/dev/add-ins/develop/itp-and-third-party-cookies
--If the reply is helpful, please Upvote and Accept as answer--