Poor Performance in Cluster Shared Volumes

Question

Hi All, 

I have noticed on several clusters upon which I've worked, that there is a huge fall off in the expected performance after LUNs added to the hosts are turned over into cluster shared volumes. With the LUNs attached to the hosts before clustering, I typically see the full performance afforded by the the network and the SAN.

However when the CSV is created on the storage, and local Virtual machines are moved into the cluster, the guests suddenly become more sluggish. Tests then reveal that the LUN's are not delivering their full performance. 
I had wondered if the management network performance then comes into play, but I'm unable to verify this after restricting associated ports to 10Gbps. Its strange however that we typically see this kind of performance degradation once the CSV's are created and none of the VM's I work on clustered seem to have stella performance now matter what the storage backing actually is.

Test scenario setup 

2 x DL380p Gen8 with 2 controllers 10Gbps

1 new san HPE 1060 SAN 4 x 10Gbe

Hosts connected to iSCSI in isolated network via 10Gbps port, with iSCSI on in its own switch.
Data Link Network not in use by Host OS.

The SAN typically gives 3 - 4 Gbps when served up as attached LUN's. Although I am not sure how this manages this speads over the iSCSI network mentioned above but however it does, and it maintains it over large files tested up to 100GB plus.

I have export some logs. In system logs, doesn't show anything.