Interpreting Xamarin.Tensorflow.Lite Output

Nathan Sokalski 4,121 Reputation points
2022-09-24T22:27:50.62+00:00

I am using Xamarin.Tensorflow.Lite to apply a model.tflite model from Azure Custom Vision Object Detection. I currently have the following code:

AssetFileDescriptor modelfiledesc = this.Assets.OpenFd("model.tflite");  
MappedByteBuffer mbb = new FileInputStream(modelfiledesc.FileDescriptor).Channel.Map(FileChannel.MapMode.ReadOnly, modelfiledesc.StartOffset, modelfiledesc.DeclaredLength);  
Interpreter tempinterpreter = new Interpreter(mbb);  
int[] shape = tempinterpreter.GetInputTensor(0).Shape();  
ByteBuffer bb = this.ByteBufferFromBitmap(bmp, shape[1], shape[2]);  
float[][][][] xxx = ArrayExtensions.CreateJagged4<float>(new[] { 1, 13, 13, 85 });  
Dictionary<Java.Lang.Integer, Java.Lang.Object> outputs = new Dictionary<Java.Lang.Integer, Java.Lang.Object>();  
outputs.Add(new Java.Lang.Integer(0), Java.Lang.Object.FromArray(xxx));  
tempinterpreter.RunForMultipleInputsOutputs(new Java.Lang.Object[] { bb }, outputs);  
float[][][][] x = outputs.Values.ToArray()[0].ToArray<float[][][]>();  

At the end of this code, I end up with a float[1][13][13][85]. This contains a bunch float values (some of which are negative). I am having trouble figuring out exactly how to translate this data into the bounding boxes, confidence values, and labels. This is partly because it has 4 dimensions (rather than the 3 that most of the stuff I have found have). I looked at the following:

https://github.com/tensorflow/tensorflow/issues/39803

But I am still trying to figure out exactly how to use it. Also, in case it matters, my model has 12 labels. Can anybody help me figure out exactly hot to translate/interpret this data? Thanks.

Azure AI Custom Vision
Azure AI Custom Vision
An Azure artificial intelligence service and end-to-end platform for applying computer vision to specific domains.
222 questions
C#
C#
An object-oriented and type-safe programming language that has its roots in the C family of languages and includes support for component-oriented programming.
10,324 questions
{count} votes