About correct set up of OpenCensus

Question

About correct set up of OpenCensus

Jona 925

Hi every one,

I have a function blob trigger based with event grid as source. I'm trying to send costom data (customDimension) to Log Analytics via App Insights

This is an example code setup

from services.blob_service import BlobService
from services.queue_service import QueueService
from opencensus.ext.azure.log_exporter import AzureLogHandler

EXPERIMENT_NAME = 'CSV Compressed'
logger = logging.getLogger(__name__)
logger.addHandler(AzureLogHandler())

def checker_fn(blob:func.InputStream, context:func.Context):

    blob_content = blob.read()
    size_mb = round(len(blob_content) / (1024 * 1024), 2)
    logger.info(f'File downloaded | {blob.name} | {size_mb:,} MB', extra={
        'custom_dimensions' : {
            'experiment' : EXPERIMENT_NAME,
            'blob' : blob.name,
            'size_mb' : size_mb, 
        }       
    })

However, while watching the logs, I see two entries of the same log message; one with the custom dimension and the other that hasn´t it.

log1

The entry that doesn't have operation_Name contains my custom data in the customDimension column. I'm following this example and this doc.

I know OpenCensus is deprecated, but this is legacy code that need to work as it is for now.

Why is this?

This my host.json file

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 6.0.0)"
  },
  "extensions": {
    "eventHubs": {
        "maxEventBatchSize" : 500,
        "minEventBatchSize" : 200,
        "maxWaitTime" : "00:0:05",
        "clientRetryOptions":{
            "mode" : "exponential",
            "tryTimeout" : "00:01:00",
            "delay" : "00:00:00.80",
            "maximumDelay" : "00:01:00",
            "maximumRetries" : 1
        }
    },
    "queues": {
      "maxPollingInterval": "00:00:02",
      "visibilityTimeout" : "00:02:00",
      "batchSize": 16,
      "maxDequeueCount": 1,
      "newBatchThreshold": 8,
      "messageEncoding": "base64"
    }
  }
}

Regards

0 comments

4 answers

Your answer

Answer 1

Jona 925

Hi @Sina Salam

Just coming here to see any update. I've tried many ways with no success. In this Github the error was reported, with no resolution

https://github.com/Azure/azure-functions-python-worker/issues/694

I really need a way to send custom dimentions to monitor my solution properly ... I think since it's no my code problem but Azure SKD, I kinldy ask for a support ticket ....

I think that custom dimensions logging should bee seamless, without any workaround. If not, is on Azure SDK ...

Regards

0 comments

Answer 2

Jona 925

@navba-MSFT can you give a hand on this?

regards

0 comments

Answer 3

Hello @Jona

Based on the solution provided this is a continuation of your previous question asking code references.

How you can modify your function_app.py to include the logger setup in function_app.py:

import azure.functions as func
import logging
from opencensus.ext.azure.log_exporter import AzureLogHandler
from function import checker_fn, splitter_fn, publisher_fn, consumer_fn, error_handler_fn
from services.env_loader import EnvLoader
# Logger configuration
EXPERIMENT_NAME = 'CSV Compressed'
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# Ensure the AzureLogHandler is only added once
if not any(isinstance(handler, AzureLogHandler) for handler in logger.handlers):
    logger.addHandler(AzureLogHandler())
event_hub_name = EnvLoader.get_value("EVENT_HUB")
schedule_setting = EnvLoader.get_value("SCHEDULE_SETTING")
app = func.FunctionApp()
@app.blob_trigger(arg_name="blob", path="landing/{name}.csv", connection="StorageConnection", source="EventGrid")
def checker(blob: func.InputStream, context: func.Context) -> None:
    checker_fn(blob=blob, context=context, logger=logger)
@app.blob_trigger(arg_name="blob", path="stage/{name}.csv", connection="StorageConnection", source="EventGrid")
def splitter(blob: func.InputStream, context: func.Context) -> None:
    splitter_fn(blob=blob, context=context, logger=logger)
@app.blob_trigger(arg_name="blob", path="rdz/{name}.csv", connection="StorageConnection", source="EventGrid")
def publisher(blob: func.InputStream, context: func.Context) -> None:
    publisher_fn(blob=blob, context=context, logger=logger)
@app.event_hub_message_trigger(arg_name="message", event_hub_name=event_hub_name, connection="EventHubConnection") 
def consumer(message: func.EventHubEvent, context: func.Context):
    consumer_fn(message=message, context=context, logger=logger)
@app.timer_trigger(arg_name="timer", run_on_startup=False, schedule=schedule_setting)
def error_handler(timer: func.TimerRequest, context: func.Context) -> None:
    error_handler_fn(timer=timer, context=context, logger=logger)

About your function.py update your function definitions to accept the logger as a parameter:

import logging
from opencensus.ext.azure.log_exporter import AzureLogHandler
import azure.functions as func
EXPERIMENT_NAME = 'CSV Compressed'
def checker_fn(blob: func.InputStream, context: func.Context, logger: logging.Logger):
    blob_content = blob.read()
    size_mb = round(len(blob_content) / (1024 * 1024), 2)
    custom_dimensions = {
        'experiment': EXPERIMENT_NAME,
        'blob': blob.name,
        'size_mb': size_mb,
    }
    # Log the message with custom dimensions
    logger.info(f'File downloaded | {blob.name} | {size_mb:,} MB', extra={'custom_dimensions': custom_dimensions})
def splitter_fn(blob: func.InputStream, context: func.Context, logger: logging.Logger):
    # Your implementation here
    pass
def publisher_fn(blob: func.InputStream, context: func.Context, logger: logging.Logger):
    # Your implementation here
    pass
def consumer_fn(message: func.EventHubEvent, context: func.Context, logger: logging.Logger):
    # Your implementation here
    pass
def error_handler_fn(timer: func.TimerRequest, context: func.Context, logger: logging.Logger):
    # Your implementation here
    pass

Regards,

Sina

Jona 925 Reputation points

2024-07-11T05:26:04.2066667+00:00

I've just realized that the data get stored properly in customDimension only just for exceptions .... it's rare

REgards
Sina Salam 28,361 Reputation points Volunteer Moderator

2024-07-11T06:44:20.8966667+00:00

Hi Jona,

It's good to see that the issues were resolved.

Kindly accept the answer to close the thread for other community member.

Regards,

Sina
Jona 925 Reputation points

2024-07-11T07:09:43.63+00:00

Hi @Sina Salam

Custom Dimension just appear properly when using logging.exception (so, all my logged exception have my custom data en theirs custom dimension). INFO messages don't have custom dimension in place, yet, which is the issue of this thread

REagrds
Jona 925 Reputation points

2024-07-18T02:32:22.7133333+00:00

Just walking around if there is any update on this

Answer 4

Sina Salam 28,361 Volunteer Moderator

Hello Jona,

Welcome to the Microsoft Q&A and thank you for posting your questions here. With kind of your detail explanations thank you once again.

Problem

I understand that you are having issue of duplicate log entries in Azure Application Insights where one entry includes the custom dimensions, and the other does not.

Solution

To address the issue of duplicate log entries in Azure Application Insights where one entry includes the custom dimensions and the other does not, I examined a few areas of your setup as posted. Below are some of the potential causes and solutions:

Ensure that the AzureLogHandler is correctly configured and attached to the logger and the custom dimensions should be properly included in the log entries sent to Application Insights. This will help to affirm the AzureLogHandler is only added once to prevent duplicate log entries and confirming that custom dimensions are consistently included in the log entries.
Your host.json configuration appears to be set up correctly for sampling settings, but it's worth verifying that there are no additional configurations that might be affecting the logging behavior.
Sometimes duplicate log entries can occur if the logger is added multiple times. Check your code to ensure that the logger is only being added once.
Since OpenCensus is deprecated, there might be some underlying issues with how it handles custom dimensions. Upgrading to the newer OpenTelemetry SDK would be ideal, but if that's not an option, you can try the following workaround to ensure the custom dimensions are consistently included.

The below is an updated example of your function with some adjustments to ensure custom dimensions are properly logged:

import logging
from opencensus.ext.azure.log_exporter import AzureLogHandler
import azure.functions as func
EXPERIMENT_NAME = 'CSV Compressed'
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# Ensure the AzureLogHandler is only added once
if not any(isinstance(handler, AzureLogHandler) for handler in logger.handlers):
    logger.addHandler(AzureLogHandler())
def checker_fn(blob: func.InputStream, context: func.Context):
    blob_content = blob.read()
    size_mb = round(len(blob_content) / (1024 * 1024), 2)
    custom_dimensions = {
        'experiment': EXPERIMENT_NAME,
        'blob': blob.name,
        'size_mb': size_mb,
    }
    # Log the message with custom dimensions
    logger.info(f'File downloaded | {blob.name} | {size_mb:,} MB', extra={'custom_dimensions': custom_dimensions})

You can also go to your Application Insights, check the logs to see if there are still duplicate entries. Use Kusto Query Language (KQL) in Application Insights to filter and inspect the log entries. For example:

   traces
   | where message contains "File downloaded"
   | order by timestamp desc

Accept Answer

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam

Jona 925

Hi @Sina Salam ,

The code I posted is from a file where the inner source code is placed. Then, from the function_app.py file I call these functions.

Should I put these lines in the main file also? This is my function_app.py

import azure.functions as func
# Importing the functions code
from function import checker_fn
from function import splitter_fn
from function import publisher_fn
from function import consumer_fn
from function import error_handler_fn
from services.env_loader import EnvLoader

event_hub_name = EnvLoader.get_value("EVENT_HUB")
schedule_setting = EnvLoader.get_value("SCHEDULE_SETTING")

app = func.FunctionApp()

@app.blob_trigger(arg_name="blob", path="landing/{name}.csv", connection="StorageConnection", source="EventGrid")
def checker(blob:func.InputStream, context: func.Context) -> None:
    checker_fn(blob=blob, context=context)

@app.blob_trigger(arg_name="blob", path="stage/{name}.csv", connection="StorageConnection", source="EventGrid")
def splitter(blob:func.InputStream, context: func.Context) -> None:
    splitter_fn(blob=blob, context=context)

@app.blob_trigger(arg_name="blob", path="rdz/{name}.csv", connection="StorageConnection", source="EventGrid")
def publisher(blob:func.InputStream, context: func.Context) -> None:
    publisher_fn(blob=blob, context=context)

@app.event_hub_message_trigger(arg_name="message", event_hub_name=event_hub_name, connection="EventHubConnection") 
def consumer(message: func.EventHubEvent, context: func.Context):
    consumer_fn(message=message, context=context)

@app.timer_trigger(arg_name="timer", run_on_startup=False, schedule=schedule_setting)
def error_handler(timer: func.TimerRequest, context: func.Context) -> None:
    error_handler_fn(timer=timer, context=context)

Regards

Jona 925 Reputation points

2024-07-10T18:23:28.0266667+00:00

It is still happening de same. Two message logs, one with operation_Name and the other not. The one with empty operation_Name is the one with my custom data in the customDimension column.

And here I have my custom data. Notice the operation_Name, it's empty

Regards
Sina Salam 28,361 Reputation points Volunteer Moderator

2024-07-10T21:19:45.9533333+00:00
Hello Jona,

Thank for your feedback.

You can go ahead to add the code.

Ensure that the logger configuration is correctly set up in your function_app.py file as well.

Then, in function.py update your function definitions to accept the logger as a parameter.

For more clarity:

Ensure that the AzureLogHandler is added only once to avoid multiple instantiations of the logger.

Pass the logger instance to each function to ensure that they use the same logger configuration.

Modify each function to accept the logger parameter and use it for logging.

The above will solve the issue.

Success,

Sina

Jona 925

Hi @Sina Salam

Excuse me, ¿can you give some quidance on this?.

Then, in function.py update your function definitions to accept the logger as a parameter.

I can't get this very well ... could you give some code snippe o depeer explanation? I don't understand "as a parameter" (obviously, I know what a parameter is)

This is the head of my file function_app.py

import logging
import azure.functions as func
from function import checker_fn
from function import splitter_fn
from function import publisher_fn
from function import consumer_fn
from function import error_handler_fn
from services.env_loader import EnvLoader
from opencensus.ext.azure.log_exporter import AzureLogHandler

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

if not any(isinstance(handler, AzureLogHandler) for handler in logger.handlers):
    logger.addHandler(AzureLogHandler())

event_hub_name = EnvLoader.get_value("EVENT_HUB")
schedule_setting = EnvLoader.get_value("SCHEDULE_SETTING")

app = func.FunctionApp()

@app.blob_trigger(arg_name="blob", path="landing/{name}.csv", connection="StorageConnection", source="EventGrid")
def checker(blob:func.InputStream, context: func.Context) -> None:
    checker_fn(blob=blob, context=context)

# more source code below ...

Pass the logger instance to each function to ensure that they use the same logger configuration. Modify each function to accept the logger parameter and use it for logging.

I'm a little lost here .. ¿How can I pass the logger as a parameter? The file where the real source code of the functions is function.py and looks like this:

import pandas
import math
import numpy
import logging
import traceback
import csv
import json
import gzip
import azure.functions as func
from common.event_hubs_exception import EventHubsException
from common.metrics_monitor_exception import MetricsMonitorException
from io import StringIO
from urllib.parse import unquote_to_bytes
from services.power_bi_service import PowerBiService
from services.event_hub_service import EventHubService
from services.metrics_monitor_service import MetricsMonitorService
from services.env_loader import EnvLoader
from services.blob_service import BlobService
from services.queue_service import QueueService
from opencensus.ext.azure.log_exporter import AzureLogHandler

EXPERIMENT_NAME = 'CSV Compressed'

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

if not any(isinstance(handler, AzureLogHandler) for handler in logger.handlers):
    logger.addHandler(AzureLogHandler())


def checker_fn(blob:func.InputStream, context:func.Context):

    blob_content = blob.read()
    size_mb = round(len(blob_content) / (1024 * 1024), 2)
    logger.info(f'File downloaded | {blob.name} | {size_mb:,} MB', extra={
        'custom_dimensions' : {
            'experiment' : EXPERIMENT_NAME,
            'blob' : blob.name,
            'size_mb' : size_mb, 
        }       
    })
    new_metadata = dict()

    try:

        # Begin: Initial set up for 'OriginalBlob' metadata
        blob_service = BlobService()
        blob_key = blob.name.split('/')[-1] # with ".csv"
        metadata = blob_service.get_blob_metadata(container='landing', blob_key=blob_key)
        original_blob = metadata.get('OriginalBlob')

# more source code below

Honestly speaking, I need to see some code to have a indea of what you are expressing. I'd appreciate your help on this.

Best regards and hoping for your assistance

Jona

Jona 925 Reputation points

2024-07-11T01:21:18.67+00:00

I don't have any progress. There two logs yet, one with the custom data and the other not

The one with operation_Name:

The one with no operation_Name:

I realized that in both cases there is some property referencing a kind of process (ProcessId and process)

I saw this link, but I don't how to "instantiate just one handler"

Regards

Jona 925

It's still not working. The same: two logs messages; one with operation_Name and the other one not. The last one contains my custom data.

I was reading some github pages

Without being and expert on logging, checking the last link, somebody found a workaround making sure that there weren't any other logging handler. So My question is, given the snippe you shared with me:

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

# Ensure the AzureLogHandler is only added once
if not any(isinstance(handler, AzureLogHandler) for handler in logger.handlers):
    logger.addHandler(AzureLogHandler())

This adds AzureLogHandler to the current list of handlers, ¿Could that be causing my issue, in case there is another handler in the logger?, ¿Would it be better to remove all handlers and add the Azure one alone?

I'm trying something like this:

import logging
import azure.functions as func
from function import checker_fn
from function import splitter_fn
from function import publisher_fn
from function import consumer_fn
from function import error_handler_fn
from services.env_loader import EnvLoader
from opencensus.ext.azure.log_exporter import AzureLogHandler

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
logger.handlers.clear()
logger.addHandler(AzureLogHandler())
# Ensure the AzureLogHandler is only added once
#if not any(isinstance(handler, AzureLogHandler) for handler in logger.handlers):
#    logger.addHandler(AzureLogHandler())

event_hub_name = EnvLoader.get_value("EVENT_HUB")
schedule_setting = EnvLoader.get_value("SCHEDULE_SETTING")

app = func.FunctionApp()
...

Regards

Jona 925 Reputation points

2024-07-11T04:56:59.44+00:00

it didn't work neither :(

Share via

About correct set up of OpenCensus

4 answers

Problem

Solution

Accept Answer

Your answer