InferenceScope Class

Provides OpenTelemetry tracing scope for generative AI inference operations.

Initialize the inference scope.

Constructor

InferenceScope(request: Request, details: InferenceCallDetails, agent_details: AgentDetails, user_details: UserDetails | None = None, span_details: SpanDetails | None = None)

Parameters

Name	Description
request Required	Request details for the inference
details Required	The details of the inference call
agent_details Required	The details of the agent making the call
user_details	Optional human user details Default value: None
span_details	Optional span configuration (parent context, timing) Default value: None

Methods

record_finish_reasons	Records the finish reasons for telemetry tracking.
record_input_messages	Records the input messages for telemetry tracking. Accepts plain strings (auto-wrapped as OTEL ChatMessage with role user) or a structured InputMessages container.
record_input_tokens	Records the number of input tokens for telemetry tracking.
record_output_messages	Records the output messages for telemetry tracking. Accepts plain strings (auto-wrapped as OTEL OutputMessage with role assistant) or a structured OutputMessages container.
record_output_tokens	Records the number of output tokens for telemetry tracking.
record_thought_process	Records the thought process.
start	Creates and starts a new scope for inference tracing.

record_finish_reasons

Records the finish reasons for telemetry tracking.

record_finish_reasons(finish_reasons: List[str]) -> None

Parameters

Name	Description
finish_reasons Required	List of finish reasons

record_input_messages

Records the input messages for telemetry tracking.

Accepts plain strings (auto-wrapped as OTEL ChatMessage with role user) or a structured InputMessages container.

record_input_messages(messages: str | list[str] | InputMessages) -> None

Parameters

Name	Description
messages Required	List of input message strings or an InputMessages container

record_input_tokens

Records the number of input tokens for telemetry tracking.

record_input_tokens(input_tokens: int) -> None

Parameters

Name	Description
input_tokens Required	Number of input tokens

record_output_messages

Records the output messages for telemetry tracking.

Accepts plain strings (auto-wrapped as OTEL OutputMessage with role assistant) or a structured OutputMessages container.

record_output_messages(messages: str | list[str] | OutputMessages) -> None

Parameters

Name	Description
messages Required	List of output message strings or an OutputMessages container

record_output_tokens

Records the number of output tokens for telemetry tracking.

record_output_tokens(output_tokens: int) -> None

Parameters

Name	Description
output_tokens Required	Number of output tokens

record_thought_process

Records the thought process.

record_thought_process(thought_process: str) -> None

Parameters

Name	Description
thought_process Required	The thought process to record

start

Creates and starts a new scope for inference tracing.

static start(request: Request, details: InferenceCallDetails, agent_details: AgentDetails, user_details: UserDetails | None = None, span_details: SpanDetails | None = None) -> InferenceScope

Parameters

Name	Description
request Required	Request details for the inference
details Required	The details of the inference call
agent_details Required	The details of the agent making the call
user_details	Optional human user details Default value: None
span_details	Optional span configuration (parent context, timing) Default value: None

Returns

Type	Description
	A new InferenceScope instance