Azure APIM doesn't support custom assemblies. Please feel free to raise a feature request to add support for this by sharing your use case.
An alternative would be to have a function app using your custom assembly that is exposed over HTTP, which you could call from your policies using the send-request policy.
Since this will be over the network, latency is something you would have to consider. You could address this in a couple of ways
- Add caching
- Deploy the function in the same region as the APIM Gateway. So, if you have multiple deployments of APIM, have a deployment of the function app in each region. Refer to this reference architecture that show cases this.