Azure Application Gateway | Rule ID 913101 Python Requests are getting blocked

sujith reddy komma 76 Reputation points
2020-07-15T06:20:14.823+00:00

Hi ,

When we are trying to send requests to APIM using Jupyters notebook in Python, We are getting blocked in the application Gateway like the below

"ruleId": "913101",
"ruleGroup": "913-SCANNER-DETECTION",
"message": "Found User-Agent associated with scripting/generic HTTP client",
"action": "Matched",
"site": "Global",
"details": {
"message": "Warning. Matched phrase \"python-requests ...\" at REQUEST_HEADERS:User-Agent.",
"data": "Matched Data: python-requests found within REQUEST_HEADERS:User-Agent: python-requests/2.22.0",
"file": "rules/REQUEST-913-SCANNER-DETECTION.conf",
"line": "156"
},
Is there a workaround for this with out disbaling any?

Thanks,
Sujith.

Azure API Management
Azure API Management
An Azure service that provides a hybrid, multi-cloud management platform for APIs.
1,739 questions
Azure Application Gateway
Azure Application Gateway
An Azure service that provides a platform-managed, scalable, and highly available application delivery controller as a service.
954 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Sam Cogan 10,082 Reputation points MVP
    2020-07-15T06:56:53.953+00:00

    You can use the WAF exclusion list to allow certain files through the firewall if they trigger this rule. You can do this by matching a certain part of the request (header, cookie or attribute name). You can find details on how to do this here.

    0 comments
  2. GitaraniSharma-MSFT 46,931 Reputation points Microsoft Employee
    2020-07-16T14:34:52.76+00:00

    Hello @sujithreddykomma-6717 ,

    The Application Gateway WAF when running in Prevention mode Blocks intrusions and attacks that the rules detect. And some of the rules can be disabled from the list per requirement.

    Since you do not want to disable any rules from the WAF rule set, you can opt for WAF exclusion lists. WAF exclusion lists allow you to omit certain request attributes from a WAF evaluation. Once an attribute is added to the WAF exclusion list, it isn't considered by any configured and active WAF rule. Exclusion lists are global in scope. And this is helpful in excluding attributes which may trigger a false positive from the WAF rules. The exclusion lists remove inspection of the field's value and some of them are as below:
    Request Headers, Request Cookies, Form field name, JSON entity & URL query string args.

    From the log shared by you, it looks like the User-Agent request header is causing the WAF to block this traffic. The user-agent request header contains a characteristic string that allows the network protocol peers to identify the application type, operating system, software vendor, or software version of the requesting software user agent. For more information, see User-Agent. In some cases, this can be legitimate traffic. So you might need to exclude this header from WAF evaluation.

    The following Azure PowerShell cmdlet excludes the user-agent header from evaluation:

    $exclusion1 = New-AzApplicationGatewayFirewallExclusionConfig -MatchVariable "RequestHeaderNames"
    -SelectorMatchOperator "Equals" `
    -Selector "User-Agent"

    For more details, please refer : https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/application-gateway-waf-configuration#waf-exclusion-lists

    Hope this helps!

    Kindly let us know if the above helps or you need further assistance on this issue.

    Please don’t forget to "Accept the answer" wherever the information provided helps you, this can be beneficial to other community members.