Custom data collection rule not applying transformKQL when saving

Question

Hi

As per the article https://learn.microsoft.com/en-us/azure/sentinel/connect-cef-syslog#avoid-data-ingestion-duplication I'm trying to update my Data Collection Rule with a transformKql query in the dataFlows section of the configuration.

I'm doing this via a PUT to the API.

When applying my change:

"dataFlows": [
            {
                "streams": [
                    "Microsoft-Syslog"
                ],
                "destinations": [
                    "la-1179667834"
                ],
                "transformKql": "source | where message has 'error'"
            }
        ],

I get a 200 response code indicating a success however when viewing the DCR it doesnt look like the transformKql has applied:

"dataFlows": [
            {
                "streams": [
                    "Microsoft-Syslog"
                ],
                "destinations": [
                    "la-1179667834"
                ]
            }
        ],

Any ideas why this may be happening?

Answer 1

@Brady Kenworthy I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer.

Issue:

Solution: Resolved by @Brady Kenworthy

Answer 2

Are you not getting any Syslog data? Did you wait 30 minutes? I cannot find any unsupported operators but I wonder if "has" is too inefficient. Is there another query that you can try?

Answer 3

Found the issue.

I was using the API version ?api-version=2021-12-01-preview for my PUT command. After changing this to ?api-version=2022-06-01 it worked perfectly.