Hello there,
While installing checks if you are clicking make this as default option.
Go to Settings, Apps, Default apps, at the bottom of the list click on Choose default apps by file type.
Scroll down to .pdf and set the default to Microsoft Edge.
Now check if it is getting changed again.
Hope this resolves your Query !!
--If the reply is helpful, please Upvote and Accept it as an answer–