Hi Shunlei Tang - Thanks for reaching out.
When specifying the parameters for generating a UserDelegationSAS, you need to specify the OID of the specified used in the signedUnauthorizedObjectId field.
Thereafter, you can use this field while creating StringToSign further.
Below is a sample code to generate a user delegation SAS and then perform a directory level operation further using Python code.