Finally I've overcome "Error while deleting key".
All I had to do was (run regedit as local administrator):
manually make myself the owner of the key (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USBSTOR\Disk&Ven_JetFlash&Prod_Transcend_32GB&Rev_1100),
give myself Full Access to it in Permissions,
apply them,
press F5 to refresh and see the underlying sub-key (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USBSTOR\Disk&Ven_JetFlash&Prod_Transcend_32GB&Rev_1100\60A44C...600FA&0) and do the same thing with it, because inheritance doesn't work here.
Then do the same with:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USBSTOR\Disk&Ven_JetFlash&Prod_Transcend_32GB&Rev_1100\60A44C...600FA&0\Properties
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USBSTOR\Disk&Ven_JetFlash&Prod_Transcend_32GB&Rev_1100\60A44C...600FA&0\Properties{3464f7a4-2444-40b1-980a-e0903cb6d912}
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USBSTOR\Disk&Ven_JetFlash&Prod_Transcend_32GB&Rev_1100\60A44C...600FA&0\Properties{3464f7a4-2444-40b1-980a-e0903cb6d912}\000A
Only after all these steps I can remove key (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USBSTOR\Disk&Ven_JetFlash&Prod_Transcend_32GB&Rev_1100) totally.
And start to work with next one...
Upd01. However, the new USB drives plug in fine and create their keys in the key (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USBSTOR) I heroically cleaned up.
Even though inheritance permissions for USBSTOR are disabled and everyone (System, Creator, ...) has got "read-only" permissions for key and sub-keys. Nobody has Full access to USBSTOR.
PS C:\Windows\system32> (Get-acl -Path HKLM:\SYSTEM\CurrentControlSet\Enum\USBSTOR).AccessToString
Creator-Owner Allow ReadKey
NT AUTHORITY\System Allow ReadKey
BUILTIN\Administrators Allow ReadKey
BUILTIN\Users Allow ReadKey
ALL APPLICATION PACKAGES Allow ReadKey
PS C:\Windows\system32> Get-acl -Path HKLM:\SYSTEM\CurrentControlSet\Enum\USBSTOR | % { $_.access }
RegistryRights : ReadKey
AccessControlType : Allow
IdentityReference : Creator-Owner
IsInherited : False
InheritanceFlags : ContainerInherit
PropagationFlags : InheritOnly
RegistryRights : ReadKey
AccessControlType : Allow
IdentityReference : NT AUTHORITY\System
IsInherited : False
InheritanceFlags : ContainerInherit
PropagationFlags : None
RegistryRights : ReadKey
AccessControlType : Allow
IdentityReference : BUILTIN\Administrators
IsInherited : False
InheritanceFlags : ContainerInherit
PropagationFlags : None
RegistryRights : ReadKey
AccessControlType : Allow
IdentityReference : BUILTIN\Users
IsInherited : False
InheritanceFlags : ContainerInherit
PropagationFlags : None
RegistryRights : ReadKey
AccessControlType : Allow
IdentityReference : ALL APPLICATION PACKAGES
IsInherited : False
InheritanceFlags : ContainerInherit
PropagationFlags : None
Nevertheless...
I don't get it..
Upd01
It seems that the USBSTOR key still keeps inheriting rights from the superior key. I don't know what to do about it. Although I have cancelled all inheritance rights and made myself the owner of the key and all sub-keys with Full Access permissions, I can't rename USBSTOR for example and SYSTEM or whoever else could create a new sub-key for new connected USB drive. And I can't rename key USBSTOR.
So the permissions I have explicitly assigned to USBSTOR do not work. I mean, I'm doing something wrong with them. What I do wrong?
So the question of MS's recommended method remains open.