That's an excellent observation, and you've perfectly diagnosed the core issue: Access-Based Enumeration (ABE) is the culprit.
To improve performance without disabling ABE, I recommend two steps:
Enable Directory Cache (DirCache): This is the most effective setting for this scenario. You can configure it via PowerShell: Set-SmbServerConfiguration -DirectoryCacheLifetime 30 -Force. This tells the server to cache directory listings longer, dramatically reducing repeated permission checks.
Verify the "ABELevel" parameter: Even though you didn't use it before, explicitly setting it on Server 2025 can help. Try Set-SmbServerConfiguration -Abalevel 1 -Force to use the newer, potentially more efficient implementation.
Start with the DirCache setting, as it's most likely to give you an immediate performance boost for your users.
Hope this gets your file server performance back on track. If this resolves the issue, please hit "Accept Answer" 😊.