Hello Jakub Świętuchowski, Thank you for posting in Q&A forum. As I understand, your machine is in a workgroup, am I right?
If so, you can try to check the certificate you installed.
Open Certmgr.msc and check if there is a EFS certificate under Certificates - Current User\Personal\Certificates container.
If so, this certificate should issue to your account (you can see the column "Issued To").
And find the same certificate on folder or file (check thumbprint).
For example:
And you can try to export this certificate with private key to one location (I mean you can back up this certificate before you delete it from personal store).
At last, you can decrypt all the folders and all the files by right clicking folder or file (or root folders) and uncheck the option "Encrypt contents to secure data".
If steps above work, and then you can delete the certificate in personal store above. I hope information above is helpful. If you have any question or concern, please feel free to let us know. Best Regards, Daisy Zhou
If the Answer is helpful, please click "Accept Answer" and upvote it.