ffmpeg smooth streaming multitrack

Question

Does anyone has a working example on how to ffmpeg stream video (from screen capturing) and audio from multiple sound cards using ISMV to Azure Media Services?
This should generate video and multi track audio on a player.

Answer 1

Hi Jean,

I have this example FFMPEG ingest for Smooth that you can try to play with. First, if you are working on Windows, you will want to list all of your DirectShow devices to get the inputs that are available. Otherwise, follow the usual ffmpeg directions for devices on other OS's.

ffmpeg -list_devices true -f dshow -i dummy

Then you can play around with this sample command line and see if you can match up your audio inputs.

Video and Audio with GoPro need to set rtbufsize to large value and I had to play around with the audio offset delay (-itsoffset) to get av sync correct.

ffmpeg -y -hide_banner -f dshow -fflags nobuffer -i audio="Headset Microphone (Logitech Stereo H650e)" -itsoffset 1.00 -f dshow -fflags nobuffer -rtbufsize 2000M -i video="GoPro Webcam" -map 0:0 -map 1:0 -c:a:0 aac -b:a:0 192k -c:v:1 libx264 -preset ultrafast -tune zerolatency -s:v:0 1280x720 -r 30-g 48 -keyint_min 48 -sc_threshold 0 -minrate:v:0 3000k -maxrate:v:0 3000k -b:v:0 3000k -f ismv -movflags isml+frag_keyframe -frag_duration 1600000 "http://<YOURCHANNEL>-nimbuspm-uswe.channel.media.azure.net/<CHANNEL_ID>/ingest.isml/Streams(video)"

I got this to work nicely with my laptop microphone, and my USB Logitec headset audio. I set one track to "english" and the other to "spanish" so it shows up in the dropdown list in the Azure Media Player demo site (ampdemo.azureedge.net). I still had to mess around with the AVSync setting (-itsoffset) a bit to make it look right.

ffmpeg -y -hide_banner -f dshow -fflags nobuffer -rtbufsize 15M -i audio="Microphone Array (Synaptics Audio)" -f dshow -fflags nobuffer -i audio="Headset Microphone (Logitech Stereo H650e)" -itsoffset 1.00 -f dshow -fflags nobuffer -rtbufsize 2000M -i video="GoPro Webcam" -map 0:a:0 -map 1:a:0 -map 2:v:0 -metadata:s:a:0 language=eng -metadata:s:a:1 language=spa -c:a:0 aac -b:a:0 192k -c:a:1 aac -b:a:1 192k -c:v:2 libx264 -preset ultrafast -tune zerolatency -s:v:0 1280x720 -r 30 -g 48 -keyint_min 48 -sc_threshold 0 -minrate:v:0 3000k -maxrate:v:0 3000k -b:v:0 3000k -f ismv -movflags isml+frag_keyframe -frag_duration 1600000 "http://<YOURCHANNEL>-nimbuspm-uswe.channel.media.azure.net/<CHANNEL_ID>/ingest.isml/Streams(video)"

To do screen recording, depending on what OS you are on, there are many ways to do that - https://stackoverflow.com/questions/6766333/capture-windows-screen-with-ffmpeg

ffmpeg -y -hide_banner -f dshow -fflags nobuffer -i audio="Headset Microphone (Logitech Stereo H650e)" -itsoffset 1.00 -f gdigrab -framerate 10 -offset_x 0 -offset_y 0 -video_size 1920x1080 -show_region 1 -i desktop -map 0:0 -map 1:0 -c:a:0 aac -b:a:0 192k -c:v:1 libx264 -preset ultrafast -tune zerolatency -s:v:0 1280x720 -r 30 -g 48 -keyint_min 48 -sc_threshold 0 -minrate:v:0 3000k -maxrate:v:0 3000k -b:v:0 3000k -f ismv -movflags isml+frag_keyframe -frag_duration 1600000 "http://<YOURCHANNEL>-nimbuspm-uswe.channel.media.azure.net/<CHANNEL_ID>/ingest.isml/Streams(video)"

UPDATED:

I created a new page of example command lines in our Javascript/Node.js SDK sample repo up here:
https://github.com/Azure-Samples/media-services-v3-node-tutorials/blob/main/Live/FFmpeg/ffmpeg_commands.md

Collected as many samples as I could locate right now. Feel free to add or suggest more if you come up with any good ones.

Answer 2

Thank you @John Deutscher (MSFT)