Custom data collection rule not applying transformKQL when saving

Brady Kenworthy 20 Reputation points
2023-05-17T07:58:15.5766667+00:00

Hi

As per the article https://learn.microsoft.com/en-us/azure/sentinel/connect-cef-syslog#avoid-data-ingestion-duplication I'm trying to update my Data Collection Rule with a transformKql query in the dataFlows section of the configuration.

I'm doing this via a PUT to the API.

When applying my change:

"dataFlows": [
            {
                "streams": [
                    "Microsoft-Syslog"
                ],
                "destinations": [
                    "la-1179667834"
                ],
                "transformKql": "source | where message has 'error'"
            }
        ],

I get a 200 response code indicating a success however when viewing the DCR it doesnt look like the transformKql has applied:

"dataFlows": [
            {
                "streams": [
                    "Microsoft-Syslog"
                ],
                "destinations": [
                    "la-1179667834"
                ]
            }
        ],

Any ideas why this may be happening?

Microsoft Sentinel
Microsoft Sentinel
A scalable, cloud-native solution for security information event management and security orchestration automated response. Previously known as Azure Sentinel.
1,057 questions
0 comments No comments
{count} votes

Accepted answer
  1. Givary-MSFT 30,676 Reputation points Microsoft Employee
    2023-05-19T04:56:24.3+00:00

    @Brady Kenworthy I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer.

    Issue:

    User's image

    Solution: Resolved by @Brady Kenworthy

    User's image

    0 comments No comments

2 additional answers

Sort by: Most helpful
  1. Andrew Blumhardt 9,841 Reputation points Microsoft Employee
    2023-05-19T04:16:53.61+00:00

    Are you not getting any Syslog data? Did you wait 30 minutes? I cannot find any unsupported operators but I wonder if "has" is too inefficient. Is there another query that you can try?

    0 comments No comments

  2. Brady Kenworthy 20 Reputation points
    2023-05-19T04:28:35.5733333+00:00

    Found the issue.

    I was using the API version ?api-version=2021-12-01-preview for my PUT command. After changing this to ?api-version=2022-06-01 it worked perfectly.

    0 comments No comments