question

woodymoo avatar image
0 Votes"
woodymoo asked sidramadoss commented

Always got same 'OutputDataConversionError.TypeConversionError' , even I remove the datetime column in output in the Synapse DW sql pool, and got same error after delete and recreated stream analystic.

Always got same 'OutputDataConversionError.TypeConversionError' , even I remove the datetime column in output in Synapse DW sql pool, and got same error after delete and recreated stream analystic.

Stream Input is event hub, get dignostic log from azure sql database. Tested pass.
Stream output is a table in azure synapse analystic DW sql pool. Tested ok.
Query is like:
SELECT
Records.ArrayValue.count as [count],
Records.ArrayValue.total as [total],
Records.ArrayValue.minimum as [minimum],
Records.ArrayValue.minimum as [maximum],
Records.ArrayValue.resourceId as [resourceId],
CAST(Records.ArrayValue.time AS datetime) as [time],
Records.ArrayValue.metricName as [metricName],
Records.ArrayValue.timeGrain as [timeGrain],
Records.ArrayValue.average as [average]
INTO
OrderSynapse
FROM
dbhub d
CROSS APPLY GetArrayElements(d.records) AS Records

the query passed the test run. but stream job got into degraded state. and got error:
Source 'dblog' had 1 occurrences of kind 'OutputDataConversionError.TypeConversionError' between processing times '2021-11-12T05:28:08.7922407Z' and '2021-11-12T05:28:08.7922407Z'.

But even I deleted the stream job, drop the [time] column in output table, remove the "CAST(Records.ArrayValue.time AS datetime) as [time], " in the query statement, and recreated a new stream job, still got same error?

Part of the Activity log:
"ErrorCategory": "Diagnostic",
"ErrorCode": "DiagnosticMessage",
"Message": "First Occurred: 11/12/2021 7:39:12 AM | Resource Name: dblog | Message: Source 'dblog' had 1 occurrences of kind 'OutputDataConversionError.TypeConversionError' between processing times '2021-11-12T07:39:12.8681135Z' and '2021-11-12T07:39:12.8681135Z'. ",
"Type": "DiagnosticMessage",


Why? is there a hidden cache I can not clean?


azure-synapse-analyticsazure-stream-analytics
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

HimanshuSinha-MSFT avatar image
0 Votes"
HimanshuSinha-MSFT answered woodymoo commented

Hello @,
Thanks for the ask and using Microsoft Q&A platform .
As Florian as pointed out this is a bug at this time and ASA team is working on this .

https://stackoverflow.com/questions/69957251/azure-stream-analytic-always-got-same-outputdataconversionerror-typeconversione

"It looks like a bug in the output adapter is provoking that issue. While the fix is rolling out, you can re-order the field list to match the column order in the destination table."

We will keep you update on this , once we hear back from the internal team .

Thanks
Himanshu


It looks like a bug in the output adapter is provoking that issue. While the fix is rolling out, you can re-order the field list to match the column order in the destination table.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks very much,
Currently I have used the databricks stream to bypass the issue.
I will try to use the Azure Stream Analytics again one or two months later..

0 Votes 0 ·
KarthikThota-6206 avatar image
0 Votes"
KarthikThota-6206 answered sidramadoss commented

I am facing similar issue in Stream Analytics. Has this issue fixed at the output adapter ?

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I am facing similar issue,what is fix ?

0 Votes 0 ·
sidramadoss avatar image sidramadoss KhushbooDubey-9496 ·

Yes, this should be fixed. Can you please share more details on what is the error you are observing? If you enable diagnostic logs, you will be able to get more detailed error messages on what caused the type conversion failures : https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-job-diagnostic-logs#send-diagnostics-to-azure-monitor-logs

0 Votes 0 ·