Get-PfxCertificate returns Access Denied

I am trying to sign some code with Set-AuthenticodeSignature. To do that I need to get the certificate from a pfx file. Get-PfxCertificate returns Access Denied, and I can't find any discussion/reference/documentation that could explain why that is happening or what it means.
> get-acl test.pfx
Directory: C:\xyz
Path Owner Access
---- ----- ------
test.pfx BUILTIN\Administrators NT AUTHORITY\SYSTEM Allow FullControl…
> ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")
True
> Get-PfxCertificate -FilePath test.pfx -Password (ConvertTo-SecureString -String (Get-Content test.txt) -AsPlainText -Force) -Verbose
Get-PfxCertificate: Access denied.
I am certain that the PFX is valid since it works in other contexts, including within the same instance of the mcr.microsoft.com/powershell:windowsservercore-1809 container, as ContainerAdministrator, but not as my custom user, which is also an administrator created like
net user $UserName /ADD /ACTIVE:YES /EXPIRES:NEVER /FULLNAME:"$Description" /PASSWORDCHG:NO /PASSWORDREQ:NO
net localgroup Administrators /add $UserName
I wouldn't expect any user permissions to be relevant when loading a cert object from a pfx file via password, but it works as the ContainerAdministrator user, so I don't know. I am able to dump the binary pfx out with Get-Content as the custom user, so I don't think it's a file permission issue, maybe some nuance about pfx files?
I have also tried
> $cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2((Resolve-Path "test.pfx"), (Get-Content test.txt))
New-Object: Exception calling ".ctor" with "2" argument(s): "Access denied."
To get basically the same error.
I'm looking for any possible explanation of why I could be getting this error, what it could mean? Or any way to get more verbose output of the error?