IMFSourceReader-ReadSample got stopped after reading 2000 frames from source camera using MFCreateSourceReaderFromURL MJPG stream

Question

when i tried to read samples for my RTSP camera using URL increasing memory while reading sample and also freezes readsample with out any response after reading more than 1500 samples

here is my code. help me to resolve this issue .. thanks in advance

int _tmain(int argc, _TCHAR* argv[])
{
	CRITICAL_SECTION        m_critsec;
	InitializeCriticalSection(&m_critsec);
	HRESULT hr = S_FALSE;
	std::ofstream outputBuffer(CAPTURE_FILENAME, std::ios::out | std::ios::binary);

	CHECK_HR(CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE),
		"COM initialisation failed.");

	CHECK_HR(MFStartup(MF_VERSION),
		"Media Foundation initialisation failed.");

	IMFSourceResolver* pSourceResolver = NULL;
	IUnknown* uSource = NULL;
	IMFMediaSource* mediaFileSource = NULL;
	IMFAttributes* pVideoReaderAttributes = NULL;
	IMFSourceReader* pSourceReader = NULL;
	IMFMediaType* pReaderOutputType = NULL, *pFirstOutputType = NULL;
	MF_OBJECT_TYPE ObjectType = MF_OBJECT_INVALID;
	DWORD mftStatus = 0;

	  // Set up the reader for the file.
	CHECK_HR(MFCreateSourceResolver(&pSourceResolver),
		"MFCreateSourceResolver failed.");

	CHECK_HR(pSourceResolver->CreateObjectFromURL(
		L"rtsp://192.168.1.11:5005/routecam",		        // URL of the source.
		MF_RESOLUTION_MEDIASOURCE,  // Create a source object.
		NULL,                       // Optional property store.
		&ObjectType,				        // Receives the created object type. 
		&uSource					          // Receives a pointer to the media source.
	),
		"Failed to create media source resolver for file.");

	CHECK_HR(uSource->QueryInterface(IID_PPV_ARGS(&mediaFileSource)),
		"Failed to create media file source.");

	CHECK_HR(MFCreateAttributes(&pVideoReaderAttributes, 2),
		"Failed to create attributes object for video reader.");

	CHECK_HR(pVideoReaderAttributes->SetGUID(MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE, MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_GUID),
		"Failed to set dev source attribute type for reader config.");

	CHECK_HR(pVideoReaderAttributes->SetUINT32(MF_SOURCE_READER_ENABLE_VIDEO_PROCESSING, 1),
		"Failed to set enable video processing attribute type for reader config.");

	CHECK_HR(pVideoReaderAttributes->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_MJPG),
		"Failed to set media sub type on source reader output media type.");

	CHECK_HR(MFCreateSourceReaderFromMediaSource(mediaFileSource, pVideoReaderAttributes, &pSourceReader),
		"Error creating media source reader.");

	CHECK_HR(MFCreateMediaType(&pReaderOutputType), "Failed to create source reader output media type.");
	CHECK_HR(pReaderOutputType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video), "Failed to set major type on source reader output media type.");
	CHECK_HR(pReaderOutputType->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_YUY2), "Failed to set media sub type on source reader output media type.");

	CHECK_HR(pSourceReader->SetCurrentMediaType((DWORD)MF_SOURCE_READER_FIRST_VIDEO_STREAM, NULL, pReaderOutputType),
		"Failed to set output media type on source reader.");

	CHECK_HR(pSourceReader->GetCurrentMediaType((DWORD)MF_SOURCE_READER_FIRST_VIDEO_STREAM, &pFirstOutputType),
		"Error retrieving current media type from first video stream.");

	std::cout << "Source reader output media type: " << GetMediaTypeDescription(pFirstOutputType) << std::endl << std::endl;

	hr = pSourceReader->SetStreamSelection(
		(DWORD)MF_SOURCE_READER_ALL_STREAMS, FALSE);

	hr = pSourceReader->SetStreamSelection(
		(DWORD)MF_SOURCE_READER_FIRST_VIDEO_STREAM, TRUE);
	// Start processing frames.
	IMFSample* pVideoSample = NULL;
	DWORD streamIndex, flags;
	LONGLONG llVideoTimeStamp = 0, llSampleDuration = 0;
	int sampleCount = 1;
	DWORD sampleFlags = 0;

	while (TRUE)
	{
		EnterCriticalSection(&m_critsec);
		
		printf("\tStream tick else\n");
		CHECK_HR(pSourceReader->ReadSample(
			MF_SOURCE_READER_FIRST_VIDEO_STREAM,
			0,                              // Flags.
			&streamIndex,                   // Receives the actual stream index. 
			&flags,                         // Receives status flags.
			&llVideoTimeStamp,              // Receives the time stamp.
			&pVideoSample                    // Receives the sample or NULL.
		), "Error reading video sample.");
		//DebugMessage(DEBUGLOG_ENABLED, L"Time for process decoding is %lld\r\n", final_time - Initial_time_ms);


		if (flags & MF_SOURCE_READERF_STREAMTICK)
		{
			printf("\tStream tick.\n");
		}
		if (flags & MF_SOURCE_READERF_ENDOFSTREAM)
		{
			printf("\tEnd of stream.\n");
			/*break;*/
		}
		if (flags & MF_SOURCE_READERF_NEWSTREAM)
		{
			printf("\tNew stream.\n");
			/*break;*/
		}
		if (flags & MF_SOURCE_READERF_NATIVEMEDIATYPECHANGED)
		{
			printf("\tNative type changed.\n");
			/*break;*/
		}
		if (flags & MF_SOURCE_READERF_CURRENTMEDIATYPECHANGED)
		{
			printf("\tCurrent type changed.\n");
			//break;
		}

		if (pVideoSample)
		{
			printf("Processing sample %i.\n", sampleCount);

			CHECK_HR(pVideoSample->SetSampleTime(llVideoTimeStamp), "Error setting the video sample time.");
			CHECK_HR(pVideoSample->GetSampleDuration(&llSampleDuration), "Error getting video sample duration.");
			CHECK_HR(pVideoSample->GetSampleFlags(&sampleFlags), "Error getting sample flags.");

			printf("Sample count %d, Sample flags %d, sample duration %I64d, sample time %I64d\n", sampleCount, sampleFlags, llSampleDuration, llVideoTimeStamp);
			
			/*CHECK_HR(WriteSampleToFile(pVideoSample, &outputBuffer),
				"Failed to write sample to file.");
*/
			sampleCount++;
		}

		DWORD cnt;

		if (pVideoSample != NULL) {
			

			SAFE_RELEASE(pVideoSample);
			
		}
		Sleep(25);
		LeaveCriticalSection(&m_critsec);
	}

	outputBuffer.close();

done:

	printf("finished.\n");
	auto c = getchar();

	SAFE_RELEASE(pSourceResolver);
	SAFE_RELEASE(uSource);
	SAFE_RELEASE(mediaFileSource);
	SAFE_RELEASE(pVideoReaderAttributes);
	SAFE_RELEASE(pSourceReader);
	SAFE_RELEASE(pReaderOutputType);
	SAFE_RELEASE(pFirstOutputType);

	return 0;
}

Answer 1

Hello Rishap,

Thank you for your question and for reaching out with your question today.

It seems like you are experiencing issues with increasing memory usage and freezing after reading more than 1500 samples from an RTSP camera in your code. There could be several potential causes for these problems. Here are a few suggestions to help resolve the issues:

1. Memory Management: Make sure you are releasing any allocated resources properly. It's important to call the Release method on COM objects to free the associated memory when you no longer need them. Check that all objects are released appropriately, including pVideoSample and other objects created throughout the code.
2. Resource Cleanup: Verify that all resources are properly cleaned up and released before exiting the application. This includes closing files (outputBuffer.close()) and releasing critical sections (DeleteCriticalSection(&m_critsec)). Failing to clean up resources can lead to memory leaks and unexpected behavior.
3. Threading and Synchronization: Ensure that the critical section (m_critsec) is used correctly for thread synchronization. The critical section should be entered before accessing shared resources and released afterward. In your code, you are using EnterCriticalSection and LeaveCriticalSection within the while loop. Make sure that other threads, if any, are not causing conflicts or blocking the execution.
4. Error Handling: Check the return values of all function calls and handle any errors appropriately. Make sure to log or display error messages to help identify any potential issues during the execution of your code.
5. Performance Optimization: If memory usage becomes a concern, consider optimizing your code to minimize memory allocations and deallocations. For example, you can reuse memory buffers instead of allocating new ones for each sample. Also, ensure that you are using the most efficient algorithms and data structures for processing the video samples.

By addressing these points and ensuring proper resource management and error handling, you can improve the stability and performance of your application. If the issues persist, further debugging and profiling may be necessary to identify any specific bottlenecks or underlying issues in the code or the RTSP camera interaction.

I used AI provided by ChatGPT to formulate part of this response. I have verified that the information is accurate before sharing it with you.

If the reply was helpful, please don’t forget to upvote or accept as answer.

Best regards.