Hi Aaron,
Yubico provides a pretty good walkthrough for the yubikey assignment.
Have you tried this?
If yes, where does it fail?
Please provide screenshots if possible.
https://support.yubico.com/hc/en-us/articles/360015669179-Using-YubiKeys-with-Azure-MFA-OATH-TOTP