Application performance Issues

Question

Hi all

Hope you lovely people can help.

The problem

We have an application called EROS which is supported by a company called IDOX, that used to take 4-5 mins to load, now is taking 30+mins to run. Also, when moving from field to field within a form it is taking around 30 secs to load.

OS = Windows 10

My findings

There is an installation of the application on a server. There are 6 users who have the application also installed on their laptops. I also noticed that on the users desktop, in the shortcut properties it is pointing to a UNC path which seems to be the application (.exe) on the server.

Users are all connected via wireless and using VPN to connect to company resources. They all have at least a 20mbps connection.

When they go into the office using their laptop and connected via a dock (wired connection) the application takes 4-5 mins to load (this is normal and always has been).

Had a user take a dock and a USB to Ethernet dongle home and tested. Application load times were 4-5 mins.

I have also been told that the application uses web services.

Our network team - They've come back and said that they haven't made any changes recently and cannot see anything that would indicate problems.

Database Guru - Has checked the database for the application which is located on another server for any performance issues and everything has come back ok/fine.

IDOX IT Support - Don't believe the issue is on their end and as they haven't made any changes or pushed any updates etc.

Server - Checked disk/cpu usage using task manager and nothing weird/suspect going on there.

What I plan on doing to troubleshoot further

Having done some 'googling' it seems like articles are pointing to using Process Monitor. Can anyone help me with how to use process monitor to troubleshoot. Also if you have any other suggestions on how to troubleshoot this I'd appreciate it.

Thank you

Update - 01-12-2020

I've managed to capture logfiles using Procmon64 and ran it before i ran the application. There are mammoth-load of logs which I've done my best to exclude out 'thing's which i don't believe are required. I'm not sure where to go from here.

@Mark Russinovich - Would REALLY appreciate your help mate please. Love your tools, but I'm a newbie to them and want to become an advance problem solving expert like you using sysinternals.

Answer 1

Can you check with the server team to see if:

1. The guest server instance (i.e. the one running the application) is currently running out of resources - memory, hard drive space, hard drive I/O, and CPU.
2. The Host server is running out of resources - same as above. Especially, if another guest has been added to same host or pool of hosts. They may need to uncouple the guest running your app so it can "float" to another host that is less busy.

Lastly, sometimes pending updates "hang", especially when resources are low on a server - have your server team check on the guest and host. Those pending updates sometimes cause massive slowdowns, especially on 2012 servers (not sure if yours if that old).

Hope this helps you troubleshoot, have a great weekend!

Answer 2

@Harro Penk - Thank your for reply. I updated my original question/post to answer the questions you had about server side. I should have added it in earlier. Any other ideas?

Answer 3

First of all: is this problem now generalized? I mean all the client now get 30 minutes to load the application?? Or do you still have some clients that can load faster?

If you still have some client that can load faster, you can compare a log for your app on a slow client with one of a quick client.

If now all the client have the same problem, then this is on the server, meaning that some Windows Update may have been applied, or something changed that now take so much time to load.

The first thing to do is adding exclusion for the antivirus on the server to the path of the application and all the related libraries (like the .net Framework if it uses .net or Java if it uses java, or anything else) and the path for the database. May be just an update of the antivirus may have caused this, so I would also try without Antivirus at all.

Then if you can compare a quick and a slow machine I would slowly go through that process.. unfortunately it is a slow process because you have to compare all the dll load from the different machines and especially how much time it is taking.

If you no longer can compare, then we must assume something changed on the server, so uninstall all the latest update until the situation get back to normal.

Another great tool for this kind of problem is Windows Performance Analyzer... with that you can see the different timings for each operation quickly and easily that with Procmon because WPA was created just for that..

HTH
-mario