As you know, Traffic Manager is a L7 load balancer at the DNS layer. Because of this, it doesn't support sticky sessions because the only thing it's doing is routing the client to your app service endpoint based on the configured profile. The main point of using Traffic Manager is the routing traffic methods you've configured and high availability/failover of your app service, e.g., having your app service deployed to both the west coast and east coast regions of the continental US.
For authentication, it's done at the app service level, not the load balancer. Any load balancer will only direct the client to the app service endpoint. However, the auth tokens should be part of the header. As you've correctly pointed out, App Service supports sticky sessions by default through the AAR Affinity cookie. You can disable this feature assuming your application in general is stateless. And since the token store is shared within the app service, they will still be authenticated if your app happens scales out to multiple instances.
In a high availability scenario, where you have your app deploy to separate regions, it isn't as elegant. In order to maintain that authentication, you'll need to configure the token store to a storage account that can be accessed by your app service in different regions. However, there is egress costs that will be incurred when accessing a storage account from a separate region.