When using an attribute store, the call is made in the context of the ADFS service account. The user doesn't need permission on the store.
The store could be a SQL database, an LDAP server, or a custom DLL. But if the information is stored on the user account in AD, you can simply extract and send it at token issuance using an issuance rule.