Hello, this is Emily.
Typically that's how it should work, unless you have multiple windows opened using that app, then you have to selct which window of that app, because Windows won't know which open window of that app you want to open. So unfortunately, Windows won't be able to just launch to that app when you have multiple windows/sessions opened on that app.
As a work around, you can ungroup those icons, so each window/session is its own icon even though they are the same app. To do so, go to Settings > Personalization > Task. Expand Task Behavior section, then next to "combine taskbar buttons and hide lablels", choose "never" or "whn taskbar is full". Then each icon is its own window/session, when you click, it won't ask you to select. The downside of that is your taskbar can get clutter real fast. But give it a try, the choice is yours, see which one works better for you.