How to preserve context.Variables when request body is ToFormUrlEncodedContent

Question

How to preserve context.Variables when request body is ToFormUrlEncodedContent

NC0202 201

Hi, our logging mechanism uses self-generate message-id to corelate inbound and outbound logs. It works perfectly but
when the backend service is ToFormUrlEncodedContent , we find the variable generated in the inbound policy cannot be relay to outbound policy and raise error.

"Expression evaluation failed. The given key was not present in the dictionary.
   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)",
"The given key was not present in the dictionary."

The sample code is like this, error occurs in outbound

<policies>  
     <inbound>  
     <set-variable name="message-id" value="@(Guid.NewGuid().ToString("N"))" />
            <log-to-eventhub logger-id="logger1" partition-id="1">@{  
                var jsonReq = new JObject(  
                             new JProperty("message-type", "request"),  
                             new JProperty("message-id", context.Variables["message-id"]))  
                 );  
                return jsonReq.ToString()   
               }  
     </log-to-eventhub>  
    </inbound>  

    
    <outbound>  
           <log-to-eventhub logger-id="logger1" partition-id="2">@{  
               var jsonRes = new JObject(  
                           new JProperty("message-type", "response"),  
                           new JProperty("message-id", context.Variables["message-id"]))  //exception on this line
                );  
                return jsonRes.ToString()   
     }  
     </log-to-eventhub>  
     <set-header name="message-id" exists-action="append">  
                <value>@(context.Variables["message-id"])</value>   //exception on this line
            </set-header>  
     <outbound>

Already read the section "IMessageBody" in the following link , but not sure how to deal with context.variables by preserveContent.

https://learn.microsoft.com/en-us/azure/api-management/api-management-policy-expressions

Need your advise.

Thank you

MuthuKumaranMurugaachari-MSFT 22,441 Reputation points Moderator

2023-03-09T15:14:59.1133333+00:00

NC0202 Thanks for posting your question in Microsoft Q&A. Can you share the full mock code snippet with sample request body? I seem to miss where you have used ToFormUrlEncodedContent and I assume you have request body as x-www-form-urlencoded.

1 answer

Your answer

MuthuKumaranMurugaachari-MSFT 22,441 Reputation points Moderator

2023-03-09T15:14:59.1133333+00:00

NC0202 Thanks for posting your question in Microsoft Q&A. Can you share the full mock code snippet with sample request body? I seem to miss where you have used ToFormUrlEncodedContent and I assume you have request body as x-www-form-urlencoded.

Answer 1

Sonny Gillissen 3,751 Volunteer Moderator

Hi NC0202

Thanks for reaching out on Microsoft Q&A!

When looking at the documentation, the variables live in the context of the operation thus is available in both inbound and backend:

https://learn.microsoft.com/en-us/azure/api-management/set-variable-policy

Due to your specific ask please share the code (be it anonymised) for deeper troubleshooting. I can add it in my testlab to perform additional analysis.

Please click “accept answer” afterwards if you think my response was helpful. For now drop additional queries and information in the comments please.

NC0202 201

Hi @Sonny Gillissen ,
Thank you for looking into this.

Here's the code we use now

<policies>
    <inbound>
        <set-variable name="policy-message-id" value="@(Guid.NewGuid().ToString("N"))" />
        <log-to-eventhub logger-id="logger-test" partition-id="0">@{
                   
                   var body = context.Request.Body?.As<string>(true);
                    var bodyReqLen = 0;
                    if (body != null )
                    {
                        bodyReqLen = body.Length;
                        if (body.Length > 1024)
                        {
                            body = body.Substring(0, 1024);
                        }
                    }
                   

                    var headers = context.Request.Headers
                                         .Where(h => h.Key != "Authorization" && h.Key != "Ocp-Apim-Subscription-Key")
                                         .Select(h => string.Format("{0}: {1}", h.Key, String.Join(", ", h.Value)))
                                         .ToArray<string>();
            
                     var headerString = (headers.Any()) ? string.Join("\r\n", headers) + "\r\n" : string.Empty;
            
                
				var jsonReq = new JObject(
                    new JProperty("policy-message-type", "request"),
                    new JProperty("policy-message-id", context.Variables["policy-message-id"])
				);

                return jsonReq.ToString() ;
                         
                }</log-to-eventhub>
        <set-header name="mysys-policy-message-id" exists-action="append">
            <value>@((string)context.Variables["policy-message-id"])</value>
        </set-header>
        <cors>
            <allowed-origins>
                <origin>*</origin>
            </allowed-origins>
            <allowed-methods>
                <method>GET</method>
                <method>POST</method>
                <method>OPTIONS</method>
                <method>DELETE</method>
                <method>PATCH</method>
                <method>PUT</method>
                <method>HEAD</method>
            </allowed-methods>
            <allowed-headers>
                <header>*</header>
            </allowed-headers>
            <expose-headers>
                <header>*</header>
            </expose-headers>
        </cors>
    </inbound>
    <backend>
        <forward-request follow-redirects="true" />
    </backend>
    <outbound>
        <set-header name="Server" exists-action="delete" />
        <set-header name="X-Powered-By" exists-action="delete" />
     
        <log-to-eventhub logger-id="logger-test" partition-id="1">@{
        
          var body = context.Response.Body?.As<string>(true);
          var bodyResLen = 0;
            if (body != null )
            {
                bodyResLen = body.Length;
                if (body.Length > 1024)
                {
                    body = body.Substring(0, 1024);
                }
            }

         
          var jsonRes = new JObject(
                    new JProperty("policy-message-type", "response"),
                    new JProperty("policy-message-id", context.Variables["policy-message-id"])
                );
                
               return jsonRes.ToString();

         
     }</log-to-eventhub>
    </outbound>
    <on-error>
        <set-header name="ErrorSource" exists-action="override">
            <value>@(context.LastError.Source)</value>
        </set-header>
        <set-header name="ErrorReason" exists-action="override">
            <value>@(context.LastError.Reason)</value>
        </set-header>
        <set-header name="ErrorMessage" exists-action="override">
            <value>@(context.LastError.Message)</value>
        </set-header>
        <set-header name="ErrorScope" exists-action="override">
            <value>@(context.LastError.Scope)</value>
        </set-header>
        <set-header name="ErrorSection" exists-action="override">
            <value>@(context.LastError.Section)</value>
        </set-header>
        <set-header name="ErrorPath" exists-action="override">
            <value>@(context.LastError.Path)</value>
        </set-header>
        <set-header name="ErrorPolicyId" exists-action="override">
            <value>@(context.LastError.PolicyId)</value>
        </set-header>
        <set-header name="ErrorStatusCode" exists-action="override">
            <value>@(context.Response.StatusCode.ToString())</value>
        </set-header>
        <base />
    </on-error>
</policies>

The exception in outbound would be like this

 "Expression evaluation failed. The given key was not present in the dictionary.\r\n   at System.ThrowHelper.ThrowKeyNotFoundException()\r\n   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)",
        "The given key was not present in the dictionary."

Thank you

NC0202 201 Reputation points

2023-03-27T05:55:39.86+00:00

sorry, please ignore this dupe comment

Sonny Gillissen 3,751 Volunteer Moderator

NC0202

No problem! I'm happy to help.

I've added your code to my test lab, and it works. However, I'm missing the part where the backend is called, so in my code I've used Microsoft Learn as backend. This works as expected:

User's image

My best guess is that the problem is not in the code used for Event Hub, but in the backend (or processing the backend response. For your reference, this is the code I've used:

<policies>
    <inbound>
        <set-variable name="policy-message-id" value="@(Guid.NewGuid().ToString("N"))" />
        <log-to-eventhub logger-id="logger-test" partition-id="0">@{
                   
                   var body = context.Request.Body?.As<string>(true);
                    var bodyReqLen = 0;
                    if (body != null )
                    {
                        bodyReqLen = body.Length;
                        if (body.Length > 1024)
                        {
                            body = body.Substring(0, 1024);
                        }
                    }
                   

                    var headers = context.Request.Headers
                                         .Where(h => h.Key != "Authorization" && h.Key != "Ocp-Apim-Subscription-Key")
                                         .Select(h => string.Format("{0}: {1}", h.Key, String.Join(", ", h.Value)))
                                         .ToArray<string>();
            
                     var headerString = (headers.Any()) ? string.Join("\r\n", headers) + "\r\n" : string.Empty;
            
                
				var jsonReq = new JObject(
                    new JProperty("policy-message-type", "request"),
                    new JProperty("policy-message-id", context.Variables["policy-message-id"])
				);

                return jsonReq.ToString() ;
                         
                }</log-to-eventhub>
        <set-header name="mysys-policy-message-id" exists-action="append">
            <value>@((string)context.Variables["policy-message-id"])</value>
        </set-header>
        <cors>
            <allowed-origins>
                <origin>*</origin>
            </allowed-origins>
            <allowed-methods>
                <method>GET</method>
                <method>POST</method>
                <method>OPTIONS</method>
                <method>DELETE</method>
                <method>PATCH</method>
                <method>PUT</method>
                <method>HEAD</method>
            </allowed-methods>
            <allowed-headers>
                <header>*</header>
            </allowed-headers>
            <expose-headers>
                <header>*</header>
            </expose-headers>
        </cors>
        <set-backend-service base-url="https://learn.microsoft.com" />
        <rewrite-uri template="/" copy-unmatched-params="false" />
    </inbound>
    <backend>
        <forward-request follow-redirects="true" />
    </backend>
    <outbound>
        <set-header name="Server" exists-action="delete" />
        <set-header name="X-Powered-By" exists-action="delete" />
        <log-to-eventhub logger-id="logger-test" partition-id="1">@{
        
          var body = context.Response.Body?.As<string>(true);
          var bodyResLen = 0;
            if (body != null )
            {
                bodyResLen = body.Length;
                if (body.Length > 1024)
                {
                    body = body.Substring(0, 1024);
                }
            }

         
          var jsonRes = new JObject(
                    new JProperty("policy-message-type", "response"),
                    new JProperty("policy-message-id", context.Variables["policy-message-id"])
                );
                
               return jsonRes.ToString();

         
     }</log-to-eventhub>
    </outbound>
    <on-error>
        <set-header name="ErrorSource" exists-action="override">
            <value>@(context.LastError.Source)</value>
        </set-header>
        <set-header name="ErrorReason" exists-action="override">
            <value>@(context.LastError.Reason)</value>
        </set-header>
        <set-header name="ErrorMessage" exists-action="override">
            <value>@(context.LastError.Message)</value>
        </set-header>
        <set-header name="ErrorScope" exists-action="override">
            <value>@(context.LastError.Scope)</value>
        </set-header>
        <set-header name="ErrorSection" exists-action="override">
            <value>@(context.LastError.Section)</value>
        </set-header>
        <set-header name="ErrorPath" exists-action="override">
            <value>@(context.LastError.Path)</value>
        </set-header>
        <set-header name="ErrorPolicyId" exists-action="override">
            <value>@(context.LastError.PolicyId)</value>
        </set-header>
        <set-header name="ErrorStatusCode" exists-action="override">
            <value>@(context.Response.StatusCode.ToString())</value>
        </set-header>
        <base />
    </on-error>
</policies>

Another possibility could be that creating the EventHub logger didn't go well. Did you follow the steps as described in this Microsoft Learn documentation?

https://learn.microsoft.com/en-us/rest/api/apimanagement/current-ga/logger/create-or-update?tabs=HTTP

Could you please try with my working code, and adding in the missing pieces for backend processing one-by-one and see where the issue occurs? You can also find this in the 'Trace' option of your APIM.

Please let me know if you've found the trouble code, so we can zoom into that :-)

Share via

How to preserve context.Variables when request body is ToFormUrlEncodedContent

1 answer

Your answer