What is a proper way decode frame ot image bitmap

Question

Dear Gurus!

I use RtspClientSharp for receiving frames from the remote Ip camera in Blazor server Application. And now I need to convert the received frame to a bitmap (imageByte in the next expression) for use as a source in the <img> tag , something as follows:

imageSrc = "Data:Image/jpg; base64," + Convert.ToBase64String(imageBytes)

Can anyone give me an example of which function can be used from OpenCVSharp and with what parameters to convert rawFrame (H.264) -> imageBytes? Or what to read and in what direction to move. The topic is new to me and the deadline is soon

Answer 1

Hi @Анатолий Терентьев，

In short, playing RTSP streams in web pages is not a standard or widely supported practice. In most cases, converting an RTSP stream to an HTTP stream is a more practical and compatible solution.

From the performance perspective of the blazor application, the Brazor Web Application is not a good choice for processing video streams, and generally the production environment will have a separate media server that only provides .m3u8 files. This makes it more compatible with other web applications or desktop applications.

I did some tests and will provide the full steps below, I hope you will be enlightened.

Step1. Since there is no remote IP camera, I used OBS RTSP Server to capture my desktop and simulate the camera data.

Step2. Open the rtsp://localhost:554/live in VLC palyer.

Step3. Using FFmpeg to convert the stream. Command like :ffmpeg -i rtsp://localhost/live -c:v libx264 -c:a aac -ac 2 -strict -2 -f hls -hls_time 4 -hls_playlist_type event test.m3u8.

Step4. Tips: I create a site in IIS to serve the .m3u8 files. I have enabled CORS, and add MIME type.

Step5. Open [http://localhost:8000/test.m3u8](http://localhost:8000/test.m3u8) in VLC player.

Step6. My Edit.cshtml

@page "/camera"
@model BlazorServerApp.Pages.CameraModel

<PageTitle>Remote IP Camera</PageTitle>

<!-- Video.js CSS -->
<link href="https://vjs.zencdn.net/7.8.4/video-js.css" rel="stylesheet">

<!-- Your view content starts here -->
<div class="container">
    <h2>Simulate a remote IP camera with my screen.</h2>
    <div>
        <video id="my-video" class="video-js vjs-default-skin" controls preload="auto" width="640" height="360" data-setup='{}'>
            <source src="http://localhost:8000/test.m3u8" type="application/x-mpegURL">
            Your browser does not support the video tag.
        </video>
    </div>
</div>

<!-- Video.js JS and HLS plugin -->
<script src="https://vjs.zencdn.net/7.8.4/video.js"></script>
<script src="https://cdn.jsdelivr.net/npm/videojs-contrib-hls@5.15.0/dist/videojs-contrib-hls.js"></script>

<!-- Initialize Video.js -->
<script>
    document.addEventListener('DOMContentLoaded', function () {
        var player = videojs('my-video');
        player.play();
    });
</script>


@{
}

Step7. My Edit.cshtml

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.RazorPages;

namespace BlazorServerApp.Pages
{
    public class CameraModel : PageModel
    {
        public void OnGet()
        {
        }
    }
}

Step8. (Optional) Add MIME type in Program.cs.

using BlazorServerApp.Data;
using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Web;
using Microsoft.AspNetCore.StaticFiles;

namespace BlazorServerApp
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var builder = WebApplication.CreateBuilder(args);

            // Add services to the container.
            builder.Services.AddRazorPages();
            builder.Services.AddServerSideBlazor();
            builder.Services.AddSingleton<WeatherForecastService>();

            var app = builder.Build();

            // Configure the HTTP request pipeline.
            if (!app.Environment.IsDevelopment())
            {
                app.UseExceptionHandler("/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                app.UseHsts();
            }
            var provider = new FileExtensionContentTypeProvider();
            provider.Mappings[".m3u8"] = "application/x-mpegURL";

            app.UseHttpsRedirection();

            //app.UseStaticFiles();
            app.UseStaticFiles(new StaticFileOptions
            {
                ContentTypeProvider = provider 
            });

            app.UseRouting();

            app.MapBlazorHub();
            app.MapFallbackToPage("/_Host");

            app.Run();
        }
    }
}

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Best regards,

Jason