Visual studio enters break mode when breakpoint is hit. Symbols loaded. Step into throws "Index not found"

Question

Issue -
After attaching debugger to a dotnetcore process (test exe [net core 3.1] which invokes dll [net core 2.1] (which i am trying to debug)), the visual studio loads symbols but as soon as breakpoint is hit, it enters break mode. It is a multi-threaded application where few breakpoints are working and I can step into the code. But for majority of breakpoints visual studio enters break mode.

The call stack is empty and pressing F10/F11 complains "Unable to step. Index not found." error.
Interestingly, for few files (within the same dll), i can debug.

I loaded the pdb in dotPeek and can see that the pdb contains the source files path within <Document> tag.

Note -
I used both VS2019 and VS2022 to debug but facing issue in both versions. Why does the VS cannot find the source files during debugging but hits breakpoint fine? The code works normally if i press F5 and expected response is produced.

What i already tried (not in same order)-

1. Reinstalling VS.
2. Deleting .suo file.
3. Clean, rebuild.
4. Deleting bin/obj and temp files from %APPDATA%
5. Close and reopen VS.
6. Adding source file path in solution properties -> Debug Source Files.
7. Unchecked Enable Just My Code from Debug -> Options -> General
8. Setting Debugging Info = Full in Build -> Advanced.
9. Build in Debug Mode.

Attachments-

Please help !!