I was able to follow these directions to get simulated temp sensor data through azure stream analytics on the edge. https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-deploy-stream-analytics?view=iotedge-2020-11 I am now trying to modify the code to put azure eye data through stream analytics to a cloud storage container. Need some help debugging. Steps to reproduce : I created an output in the stream analytics editor I modified the query to select * from the input stream from the azure eye and then place into the cloud storage container. SELECT * INTO [name of cloud storage container] FROM [name of input - Edge Hub] In the IOT Hub I added in Edge Devices, set module, routes the following route FROM /messages/modules/azureeyemodule/outputs/* INTO BrokeredEndpoint("/modules/streamonedgeversion2/inputs/[name of input to ASA]") In Percept Studio I then verify via view Telemetry that the Percept is at least sending data from AzureEye and the temp simulator up to cloud. I opened Azure Storage Explorer and took a look at the Storage container and I don't see the new container being populated with new data from this query. @Mike McCoy - MSFT @Amira Youssef

@QuantumCache @Amira Youssef I was able to follow the following directions and now have ASA running and outputting simulated sensor data on the edge through stream analytics and routed up through IoT Hub and can view telemetry data in Percept Studio and also see the screen shot that ASA is processing the data as shown in attached screenshot. Next step, see if I can route Percept Data through the same path. https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-deploy-stream-analytics?view=iotedge-2020-11

Stream Analytics on the Edge with Azure Percept - Debugging route

Charles Elwood 106 MVP

I was able to follow these directions to get simulated temp sensor data through azure stream analytics on the edge.
https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-deploy-stream-analytics?view=iotedge-2020-11

I am now trying to modify the code to put azure eye data through stream analytics to a cloud storage container.
Need some help debugging.

Steps to reproduce :

I created an output in the stream analytics editor
I modified the query to select * from the input stream from the azure eye and then place into the cloud storage container.
SELECT
*
INTO
[name of cloud storage container]
FROM
[name of input - Edge Hub]
In the IOT Hub I added in Edge Devices, set module, routes the following route
FROM /messages/modules/azureeyemodule/outputs/* INTO BrokeredEndpoint("/modules/streamonedgeversion2/inputs/[name of input to ASA]")
In Percept Studio I then verify via view Telemetry that the Percept is at least sending data from AzureEye and the temp simulator up to cloud.
I opened Azure Storage Explorer and took a look at the Storage container and I don't see the new container being populated with new data from this query.

@Mike McCoy - MSFT @Amira Youssef

Charles Elwood 106 Reputation points MVP

2021-07-23T14:43:19.66+00:00

@Mike McCoy - MSFT I was able to go into IOT Hub -> View modules -> click on this module and click on troubleshoot. Error message excerpt below from the log.

{\"Source\":\"temp\",\"Type\":\"DataError\",\"DataErrorType\":\"InputDeserializerError.MissingColumns\",\"BriefMessage\":\"Could not deserialize the input event(s) from resource 'azureeyemodule' as Json. Some possible reasons: 1) Malformed events 2) Input source configured with incorrect serialization format\",\"ErrorCode\":\"InputDeserializationError\",\"ErrorCategory\":\"DataError\",\"Message\":\"Error while deserializing input message Id: azureeyemodule. Hit following error: Missing fields specified in query or in create table.\nFields expected: TIMECREATED. \nFields found: NEURAL_NETWORK. \nMissing fields: TIMECREATED\",\"ExampleEvents\":\"[\"{\\"NEURAL_NETWORK\\": [{\\"bbox\\": [0.161, 0.481, 0.550, 0.990],\\"label\\": \\"person\\", \\"confidence\\": \\"0.924805\\", \\"timestamp\\": \\"1627051174312572427\\"}, {\\"bbox\\": [0.157, 0.521, 0.321, 0.755],\\"label\\": \\"potted plant\\", \\"confidence\\": \\"0.806152\\", \\"timestamp\\": \\"1627051174312572427\\"}]}\"]\",\"FromTimestamp\":\"2021-07-23T14:39:36.1965051Z\",\"ToTimestamp\":\"2021-07-23T14:39:36.2107964Z\",\"EventCount\":1}"]
Charles Elwood 106 Reputation points MVP

2021-07-24T16:06:46.437+00:00

@Mike McCoy - MSFT @Jussi Niemela @Amira Youssef @HimanshuSinha-msft
I went and dug into this error a little bit

https://learn.microsoft.com/en-us/azure/stream-analytics/data-errors

The query is just to push * into the output.

It looks like it can't find certain query parameters, but that doesn't make sense with the "*".

I'm wondering now if my stream analytics modification to the query isn't being picked up and deployed?
If I modify the query, should the new query automatically get saved to the container storage and then will the Percept pick up the update automatically?
I'll try creating a completely new Stream Analytics module, delete the old one, and deploy the new one to test.
Now that I can find the error logs in the deployment section of the IOT Hub, that should help me debug this further.
QuantumCache 20,186 Reputation points

2021-07-26T15:54:49.177+00:00

Hello @Charles Elwood Community SME's on this topic or our team will review your scenario and circle back at the possible earliest time.
QuantumCache 20,186 Reputation points

2021-07-27T02:54:50.737+00:00

Hello @Charles Elwood Did you tired the other way with simulated data or without the Azure Percept to ingest telemetry data? Such as simulated sensor data from an IoT-Edge or web simulator for quick testing?

Also, you will need to get the raw message from IoT Hub and check if the payload is in valid JSON format. This error is usually caused by two readers are reading data from the same consumer group, just-in-case better to check in this direction. Ref

Please also read this StackOverflow post: https://stackoverflow.com/a/37277134/13174899
Charles Elwood 106 Reputation points MVP

2021-08-12T21:44:37.633+00:00

@QuantumCache @Mike McCoy - MSFT @Amira Youssef
Apologies for the delay. I didn't realize there were responses.
I'll check into this and get back to you tomorrow.

I l had tried the simulated temperature module before I used the actual data from the Percept but I don't think I kept records.
I'll do this again. Are there any particular logs from particular modules that I should be saving a copy of?

1 answer

Charles Elwood 106 Reputation points MVP

2021-08-16T13:02:53.467+00:00

@QuantumCache @Amira Youssef

I was able to follow the following directions and now have ASA running and outputting simulated sensor data on the edge through stream analytics and routed up through IoT Hub and can view telemetry data in Percept Studio and also see the screen shot that ASA is processing the data as shown in attached screenshot.

Next step, see if I can route Percept Data through the same path.

https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-deploy-stream-analytics?view=iotedge-2020-11
Please sign in to rate this answer.

1 person found this answer helpful.
Charles Elwood 106 Reputation points MVP

2021-08-16T13:14:06.013+00:00

Just tried to change everything over to Percept data and I am getting the same error : @QuantumCache @Amira Yousif
Sign in to comment

Share via

Stream Analytics on the Edge with Azure Percept - Debugging route

1 answer