Registering CRYPT_OID_OPEN_STORE_PROV_FUNC

Question

Hi all. I am developing a custom CSP and I need to extend the CryptoAPI to manage a custom store. In the "DllRegisterServer" function I call "CryptRegisterOIDFunction" to register my custom "CertDllOpenStoreProv" function and then I call "CertRegisterPhysicalStore" to register a physical store inside "MY" store. All functions returns OK but, after register the DLL, my function "CertDllOpenStoreProv" is never called when I open the Certificates dialog, list the certificates with "certutil", et cetera. Here is the code:

__declspec(dllexport) BOOL WINAPI CertDllOpenStoreProv(
    IN LPCSTR lpszStoreProvider,
    IN DWORD dwEncodingType,
    IN HCRYPTPROV hCryptProv,
    IN DWORD dwFlags,
    IN const void *pvPara,
    IN HCERTSTORE hCertStore,
    IN OUT PCERT_STORE_PROV_INFO pStoreProvInfo
) {
    logMessage("CertDllOpenStoreProv(lpszStoreProvider='%s', dwEncodingType=0x%llx, hCryptProv=0x%llx, dwFlags=0x%llx, pvPara=%p, hCertStore=0x%llx, pStoreProvInfo=%p)\r\n", (char *) lpszStoreProvider, (unsigned long long) dwEncodingType, (unsigned long long) hCryptProv, (unsigned long long) dwFlags, pvPara, (unsigned long long) hCertStore, pStoreProvInfo);
    return FALSE;
}

#define  SYSTEM_STORE_NAME    L"MY"
#define  PHYSICAL_STORE_NAME  L"My Physical Store"
#define  MY_OID               "MyCSP_CertDllOpenStoreProv"
#define  DLL_NAME             L"MyCSP.dll"


__declspec(dllexport) HRESULT DllRegisterServer() {
    logMessage("DllRegisterServer\r\n");
    if (CryptRegisterOIDFunction(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, CRYPT_OID_OPEN_STORE_PROV_FUNC, MY_OID, DLL_NAME, "CertDllOpenStoreProv")) {
        logMessage("    OID function for open store registered\r\n");
        CERT_PHYSICAL_STORE_INFO storeInfo;
        memset(&storeInfo, 0, sizeof(CERT_PHYSICAL_STORE_INFO));
        storeInfo.cbSize = sizeof(CERT_PHYSICAL_STORE_INFO);
        storeInfo.pszOpenStoreProvider = MY_OID;
        storeInfo.dwFlags = CERT_PHYSICAL_STORE_ADD_ENABLE_FLAG;
        storeInfo.OpenParameters.pbData = NULL;
        storeInfo.OpenParameters.cbData = 0;
        storeInfo.dwPriority = 1;
        storeInfo.dwOpenEncodingType = X509_ASN_ENCODING | PKCS_7_ASN_ENCODING;
        if (CertRegisterPhysicalStore(SYSTEM_STORE_NAME, CERT_SYSTEM_STORE_CURRENT_USER, PHYSICAL_STORE_NAME, &storeInfo, NULL)) {
            logMessage("    Physical store registered\r\n");
            return S_OK;
        }
        else {
            logMessage("    Error: Unable to register physical store\r\n");
            return E_FAIL;
        }
    }
    else {
        logMessage("    Error: Unable to register OID function for open store\r\n");
        return E_FAIL;
    }
}


__declspec(dllexport) HRESULT DllUnregisterServer() {
    logMessage("DllUnregisterServer\r\n");
    if (CertUnregisterPhysicalStore(SYSTEM_STORE_NAME, CERT_SYSTEM_STORE_CURRENT_USER, PHYSICAL_STORE_NAME)) {
        logMessage("    Physical store unregistered\r\n");
        if (!CryptUnregisterOIDFunction(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, CRYPT_OID_OPEN_STORE_PROV_FUNC, MY_OID)) {
            if (ERROR_FILE_NOT_FOUND != GetLastError()) {
                logMessage("    Error: unable to unregister OID function for open store\r\n");
                return E_FAIL;
            }
        }
        logMessage("    OID function for open store unregistered\r\n");
        return S_OK;
    }
    else {
        logMessage("    Error: unable to unregister physical store\r\n");
        return E_FAIL;
    }
}

What am I doing wrong? I have tested with both CURRENT USER and LOCAL MACHINE flags, and all ASN ENCODING combinations with same results. Any ideas?

Thank you.

Answer 1

Hello Avelino,

Thank you for your question and for reaching out with your question today.

Based on the code you provided, it seems that you are correctly registering your custom store provider and physical store. However, there could be a few potential issues that could prevent your CertDllOpenStoreProv function from being called:

1. Ensure your custom CSP DLL (MyCSP.dll) is registered properly:
   • Make sure that you are running the registration process with administrative privileges.
   • Check that the DLL is located in a directory where it can be found by the system. Placing it in the same directory as your code or in a system directory (e.g., C:\Windows\System32) should work.
   • Double-check that the DLL name (DLL_NAME) matches the actual name of your custom CSP DLL.
2. Verify that the DllRegisterServer function is being called:
   • You can add additional logging or debugging statements to check if the DllRegisterServer function is being executed.
   • Ensure that the function is properly exported from your DLL and can be accessed by the registration process.
3. Confirm the context in which you are attempting to use the custom store:
   • When using the Certificates dialog or certutil, make sure you are selecting the appropriate store and provider that should trigger the call to CertDllOpenStoreProv.
   • The custom store provider (MY) and physical store name (My Physical Store) should match the values you registered.

By checking the above points and troubleshooting any potential issues, you should be able to identify why your CertDllOpenStoreProv function is not being called.

I used AI provided by ChatGPT to formulate part of this response. I have verified that the information is accurate before sharing it with you.

If the reply was helpful, please don’t forget to upvote or accept as answer.

Answer 2

Hi all,

I have solved the problem. These are the fixes:

• In the call to "CryptRegisterOIDFunction" I must to pass 0 as first parameter.
• I must to initialize the "storeInfo.dwOpenFlags" field to "CERT_SYSTEM_STORE_USERS".

With these changes, the "CryptRegisterOIDFunction" is called correctly when user open "certmgr" or try to list the system certificates with any other software.

Thank you!

Answer 3

I have developed a custom CSP DLL, I tried pushing a certficate to window store with private key referencing to custom CSP .

I have registered my CSP in registry entry and signed and pasted the DLL in system32 and sysWOW64

When i try to sign a document using this certificate, my custom CSP DLL is not called,

but when I try to do the same (tried to access the private key and use cryptsignmessage)in a console app it is calling my dll.

Please help me out, why my csp is not called by adobe.