What is a proper way decode frame ot image bitmap

A Terentiev 85 Reputation points
2023-12-04T08:02:57.6066667+00:00

Dear Gurus!

I use RtspClientSharp for receiving frames from the remote Ip camera in Blazor server Application. And now I need to convert the received frame to a bitmap (imageByte in the next expression) for use as a source in the <img> tag , something as follows:

imageSrc = "Data:Image/jpg; base64," + Convert.ToBase64String(imageBytes)

Can anyone give me an example of which function can be used from OpenCVSharp and with what parameters to convert rawFrame (H.264) -> imageBytes? Or what to read and in what direction to move. The topic is new to me and the deadline is soon

Blazor
Blazor
A free and open-source web framework that enables developers to create web apps using C# and HTML being developed by Microsoft.
1,499 questions
{count} votes

Accepted answer
  1. JasonPan - MSFT 4,811 Reputation points Microsoft Vendor
    2023-12-07T10:11:50.5066667+00:00

    Hi @Анатолий Терентьев

    In short, playing RTSP streams in web pages is not a standard or widely supported practice. In most cases, converting an RTSP stream to an HTTP stream is a more practical and compatible solution.

    From the performance perspective of the blazor application, the Brazor Web Application is not a good choice for processing video streams, and generally the production environment will have a separate media server that only provides .m3u8 files. This makes it more compatible with other web applications or desktop applications.

    I did some tests and will provide the full steps below, I hope you will be enlightened.

    Animation

    Step1. Since there is no remote IP camera, I used OBS RTSP Server to capture my desktop and simulate the camera data.

    Step2. Open the rtsp://localhost:554/live in VLC palyer.

    User's image

    Step3. Using FFmpeg to convert the stream. Command like :ffmpeg -i rtsp://localhost/live -c:v libx264 -c:a aac -ac 2 -strict -2 -f hls -hls_time 4 -hls_playlist_type event test.m3u8.

    User's image

    Step4. Tips: I create a site in IIS to serve the .m3u8 files. I have enabled CORS, and add MIME type.

    User's image

    Step5. Open [http://localhost:8000/test.m3u8](http://localhost:8000/test.m3u8) in VLC player.

    User's image

    Step6. My Edit.cshtml

    @page "/camera"
    @model BlazorServerApp.Pages.CameraModel
    
    <PageTitle>Remote IP Camera</PageTitle>
    
    <!-- Video.js CSS -->
    <link href="https://vjs.zencdn.net/7.8.4/video-js.css" rel="stylesheet">
    
    <!-- Your view content starts here -->
    <div class="container">
        <h2>Simulate a remote IP camera with my screen.</h2>
        <div>
            <video id="my-video" class="video-js vjs-default-skin" controls preload="auto" width="640" height="360" data-setup='{}'>
                <source src="http://localhost:8000/test.m3u8" type="application/x-mpegURL">
                Your browser does not support the video tag.
            </video>
        </div>
    </div>
    
    <!-- Video.js JS and HLS plugin -->
    <script src="https://vjs.zencdn.net/7.8.4/video.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/videojs-contrib-hls@5.15.0/dist/videojs-contrib-hls.js"></script>
    
    <!-- Initialize Video.js -->
    <script>
        document.addEventListener('DOMContentLoaded', function () {
            var player = videojs('my-video');
            player.play();
        });
    </script>
    
    
    @{
    }
    
    

    Step7. My Edit.cshtml

    using Microsoft.AspNetCore.Mvc;
    using Microsoft.AspNetCore.Mvc.RazorPages;
    
    namespace BlazorServerApp.Pages
    {
        public class CameraModel : PageModel
        {
            public void OnGet()
            {
            }
        }
    }
    
    

    Step8. (Optional) Add MIME type in Program.cs.

    using BlazorServerApp.Data;
    using Microsoft.AspNetCore.Components;
    using Microsoft.AspNetCore.Components.Web;
    using Microsoft.AspNetCore.StaticFiles;
    
    namespace BlazorServerApp
    {
        public class Program
        {
            public static void Main(string[] args)
            {
                var builder = WebApplication.CreateBuilder(args);
    
                // Add services to the container.
                builder.Services.AddRazorPages();
                builder.Services.AddServerSideBlazor();
                builder.Services.AddSingleton<WeatherForecastService>();
    
                var app = builder.Build();
    
                // Configure the HTTP request pipeline.
                if (!app.Environment.IsDevelopment())
                {
                    app.UseExceptionHandler("/Error");
                    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                    app.UseHsts();
                }
                var provider = new FileExtensionContentTypeProvider();
                provider.Mappings[".m3u8"] = "application/x-mpegURL";
    
                app.UseHttpsRedirection();
    
                //app.UseStaticFiles();
                app.UseStaticFiles(new StaticFileOptions
                {
                    ContentTypeProvider = provider 
                });
    
                app.UseRouting();
    
                app.MapBlazorHub();
                app.MapFallbackToPage("/_Host");
    
                app.Run();
            }
        }
    }
    
    

    If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    Best regards,

    Jason

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful