Repeated Camera Captures

Question

I am using CaptureElement & MediaCapture for the camera, and I want to automatically continuously capture images that I will process (until the user stops the camera, of course). I want to take a picture, process it, take another, process it, etc. I do not need to keep the images once they have been processed, they are only temporary while processing. However, I am not sure how to do this. I am receiving Exceptions about the file being in use and things needing initialized (I don't know if these things are related or not). I have used CaptureElement & MediaCapture plenty of times before, but the idea of repeated captures is new for me. What would be the best way to do this? Thanks.

Here is the code involved in my problem. The XAML elements referenced in this are ceCard (a CaptureElement), imgCropped (an Image), and btnStartStopScanning (a Button). The VB.NET code (in MainPage.xaml.vb) is:

Public NotInheritable Class MainPage : Inherits Page
 Private ReadOnly endpt As New CustomVisionPredictionClient() With {.ApiKey = App.PredictionKey, .Endpoint = App.EndPoint}
 Private cardcapture As MediaCapture

 Public Sub New()
 InitializeComponent()
 End Sub

 Private Async Sub MainPage_Loaded(sender As Object, e As RoutedEventArgs) Handles Me.Loaded
 'Prepare for previewing
 Try
 Me.cardcapture = New MediaCapture()
 Await Me.cardcapture.InitializeAsync(New MediaCaptureInitializationSettings() With {.StreamingCaptureMode = StreamingCaptureMode.Video, .VideoDeviceId = (Await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture)).FirstOrDefault(Function(di) di.EnclosureLocation.Panel = Windows.Devices.Enumeration.Panel.Back).Id})
 Catch ex As Exception
 End Try
 Try
 Me.ceCard.Source = Me.cardcapture
 Catch ex As Exception
 End Try
 End Sub

 Private Async Sub btnStartStopScanning_Click(sender As Object, e As RoutedEventArgs) Handles btnStartStopScanning.Click
 Select Case Me.cardcapture.CameraStreamState
 Case CameraStreamState.NotStreaming
 Await Me.cardcapture.StartPreviewAsync()
 Me.btnStartStopScanning.Content = "Stop Scanning"
 Try : Me.ScanCard()
 Catch ex As Exception
 End Try
 Case CameraStreamState.Streaming
 Await Me.cardcapture.StopPreviewAsync()
 Me.btnStartStopScanning.Content = "Start Scanning"
 End Select
 End Sub

 Private Async Sub ScanCard()
 'Create a temporary file for the image from the camera
 Dim sf As StorageFile = Await ApplicationData.Current.LocalFolder.CreateFileAsync("originalcardimage.jpeg", CreationCollisionOption.ReplaceExisting).AsTask()
 'Capture the image from the camera
 Await Me.cardcapture.CapturePhotoToStorageFileAsync(ImageEncodingProperties.CreateJpeg(), sf)
 'Get the size of the image
 Dim imagewidth As Double = CType(Me.cardcapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.Photo), VideoEncodingProperties).Width
 Dim imageheight As Double = CType(Me.cardcapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.Photo), VideoEncodingProperties).Height
 Await Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal, Async Sub()
 Me.GenerateCard(Await Me.endpt.DetectImageWithNoStoreAsync(App.ProjectId, App.PublishedName, File.OpenRead(Path.Combine(ApplicationData.Current.LocalFolder.Path, "originalcardimage.jpeg"))), imagewidth, imageheight)
 If Me.cardcapture.CameraStreamState = CameraStreamState.Streaming Then Me.ScanCard()
 End Sub)
 End Sub

 Private Async Sub GenerateCard(result As ImagePrediction, imagewidth As Double, imageheight As Double)
 If result.Predictions.Any(Function(pm) pm.TagName = "ColorCard" AndAlso pm.Probability >= 0.8) Then
 'Get the card with the highest probability
 Dim pm As PredictionModel = result.Predictions.OrderByDescending(Function(model) model.Probability)(0)
 'Crop the image from the camera
 Dim croppedbmp As WriteableBitmap = Await BitmapFactory.FromStream(File.OpenRead(Path.Combine(ApplicationData.Current.LocalFolder.Path, "originalcardimage.jpeg")))
 croppedbmp = croppedbmp.Crop(CInt(pm.BoundingBox.Left), CInt(pm.BoundingBox.Top), CInt(pm.BoundingBox.Width), CInt(pm.BoundingBox.Height))
 Me.imgCropped.Source = croppedbmp
 End If
 End Sub
End Class

The most significant parts of this code are the ScanCard() and GenerateCard(...) methods where I call File.OpenRead(...). My basic goal is to repeat the process of capturing an image (like I do with CapturePhotoToStorageFileAsync(...)) and then looking at it (like I do in GenerateCard(...)). I think calling File.OpenRead(...) in multiple places is causing the problem, but I am not sure where or how to close (or whatever I need to do) to avoid the following Exception:

Exception thrown: 'System.IO.FileLoadException' in System.Private.CoreLib.dll
WinRT information: The file is in use. Please close the file before continuing.

What do I need to change or add? Thanks.

---

Update -1:

You are correct in that I (supposedly) do not need to create a file, but I am not sure what the process would be to get directly from the CaptureElement to a SoftwareBitmap (or, in my case, it is actually a WriteableBitmap that I need). You will notice that the method I use to capture the image is CapturePhotoToStorageFileAsync (which requires a StorageFile, which requires a file). You will also notice that after opening the file (in the GenerateCard method) I convert it to a WriteableBitmap, which is when I do stuff with it. If I had a way to get from the CaptureElement to a WriteableBitmap without using a file, that would be great, but I am not sure how to do that. I noticed that there is a CapturePhotoToStreamAsync method, but I was not quite sure how to use this (streams have always been one of my weak points, I'm never sure if I created them correctly, whether I closed them correctly, if I am using them correctly, etc.). If you could show me the best way to get from a CaptureElement to a WriteableBitmap without using a file (if using CapturePhotoToStreamAsync is the right way, can you show me the required steps so that I don't forget anything?), it would be greatly appreciated. Thanks!

---

Update -2:

I think that you have mostly answered my question, although now I have an (almost opposite question). I created the following method to capture a WriteableBitmap:

Private Async Function CaptureWriteableBitmap() As Task(Of WriteableBitmap)
 Dim softwarebmp As SoftwareBitmap = (Await (Await Me.cardcapture.PrepareLowLagPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Bgra8))).CaptureAsync()).Frame.SoftwareBitmap
 Dim writeablebmp As WriteableBitmap = New WriteableBitmap(softwarebmp.PixelWidth, softwarebmp.PixelHeight)
 softwarebmp.CopyToBuffer(writeablebmp.PixelBuffer)
 Return writeablebmp
End Function

However, you may have noticed that in my original posting (on line 45) I call the DetectImageWithNoStoreAsync method (this is a method from Azure's CustomVision service), which is where I originally used the File.OpenRead method to get a FileStream (the 3rd parameter of DetectImageWithNoStoreAsync is of type Stream). So I now need to figure out how to get from a WriteableBitmap to a Stream. I have tried methods such as WriteableBitmap.ToStream, but couldn't seem to get that to work. I am not quite sure how to get the Stream to pass (or even what kind of stream to pass, since it is obviously not a FileStream). What kind of Stream should I be passing, and how can I get it from the WriteableBitmap (or the SoftwareBitmap that I create in my CaptureWriteableBitmap method)? Thanks!

Answer 1

Hello,​

Welcome to our Microsoft Q&A platform!

According to our discussion, for take photos continuously without creating a local file, you can convert the photos to SoftwareBitmap, the specific content is in this document.

If you just need WriteableBitmap, you don't necessarily need to stream. After obtaining SoftwareBitmap, we can directly convert to WriteableBitmap.

Dim bitmap As WriteableBitmap = New WriteableBitmap(captureBitmap.PixelWidth, captureBitmap.PixelHeight)  
captureBitmap.CopyToBuffer(bitmap.PixelBuffer)  

Thanks.