Log Analytics - structured logging columns not being created?

Question

Hi,

We run some ASP.NET Core apps in Azure Container Apps, and use Serilog for structured logging to Log Analytics.

Log Analytics, usually, helpfully creates columns for the logged properties - however this appears to have stopped working since 7th October at 2am UTC?

Here's an example query that we use to view errors:

You can see that we use a number of these columns in our projections.

I noticed this morning that these weren't being populated, so performed a search to see when Log__m_s was last not empty and it shows 07/10/2023 @ 02:11:46 UTC time.

I can see that Log_s still contains the relevant JSON, with the properties and values that I'm interested in. We haven't deployed any changes to our apps for a couple of weeks now.

Any idea what's going on?

Thanks in advance,

Nat

Answer 1

I raised a support request for this and received the following response -

As of October 2023, there was a change introduced to Container Apps logging that impacts customers who dynamically log JSON output to Log Analytics.
Container Apps no longer logs separate columns for the custom properties defined in the customer's JSON output.

In the meantime, there is a work around of the issue by using the parse_json function when you query logs.

Here's a basic example of how you might write a Kusto Query Language (KQL) query to parse the JSON data from that column:

> ContainerAppsConsoleLogs_CL
> | extend ParsedJSON = parse_json(Log_s)
> | project
> LogMessage = ParsedJSON.Log_message_s,
> LogLevel = ParsedJSON.Log_level_s,
> LogTraceId = ParsedJSON.Log_trace_id_g
> 
> ```

>   
> Assuming Log_s that contains JSON formatted text like this:
> 
```json
> {"Log_message_s": "Example message","Log_level_s": "Information","Log_trace_id_g": "example-trace-id"}
> 
> ```

>   
> We are working on developing a fix to allow opting into dynamically generated log columns from JSON data through a config option, to be rolled out in the near future.


We're using a similar KQL query to the one suggested above and that seems to work well for now. Hopefully, this helps someone else too.