Hello, this is Emily.
The context here is not really clear.
Yes, Yubico key does use non-cloneable signal. But it is just another type of 2FA method.
I don't know how you are planning on using it for your device, but it is not meant for logging into your Windows if you are home users/using Microsoft account to sign into your Windows. It, however, can be used on websites using Microsoft account to sign in, such as https://account.live.com.
So, when you talked about a compromised device, that depends on how you define the compromise itself. A Yubico key will not protect you from Windows login compromise if you are using a Microsoft account. If you are using a local account, then absolutely yes. If you are talking about phishing/malware attack, and the attacker gains your Microsoft account password for websites that use your Microsoft account, the key can definitely protect, but it is no difference than other 2FA method, such as obtaining the approval code on an authentication app on your phone, etc.
Whether it is worth it is a very subjective question. If you use Microsoft account to sign into your Windows and it is this part you hope to strengthen the security, then it is not worth it.