Partager via

Camera Capture Performance (Part1)

  • Article

Still Image Capture

 

Windows Mobile 5.0 contains an Image Sink Filter (a DShow filter). This filter encodes the image data and writes the encoded image to a file. For encoding, the Image Sink Filter uses Imaging API to access the installed Imaging encoders. The still image encoders which Microsoft ships with Windows Mobile 5.0 are not optimized and should, in general be replaced by OEMs with optimized encoders for the particular platform. One limitation of Imaging framework is that it only accepts RGB input, i.e., the raw buffer sent to Imaging encoder must be in RGB format. For this very reason, the Image Sink filter does not accept YCbCr.

Besides accepting RGB data, the Image Sink Filter also accepts pre-encoded JPG data. This is useful when there is an optimized still image encoder available on a given platform, which accepts YCbCr. For this to work the actual encoding is done by the driver and the camera driver exposes a format with SubFormat field of CSDATARANGE equal to MEDIASUBTYPE_IJPG GUID. The encoded buffer will then be passed down to the Image Sink Filter simply for file I/O. This optimized encoding can also be done in a DShow Transform filter with the transform filter exposing MEDIASUBTYPE_IJPG on its output pin.

The OEMs can still use the encoders that are shipped with Windows Mobile 5.0. These encoders provide a stable, platform independent solution. However, as the resolution of still image increases, the performance of these encoders deteriorates.

 

Following is how you would change the sample null camera driver to provide JPG data on STILL pin.

 

Step 1: Define the format

In adapterprops.h define the following

#define FOURCC_IJPG mmioFOURCC('I', 'J', 'P', 'G')

#define MEDIASUBTYPE_IJPG {0x47504A49, 0x0000, 0x0010, 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}

#define DX 176

#define DY 144

#define DBITCOUNT 16

#define FRAMERATE 15

CS_DATARANGE_VIDEO DCAM_StreamMode =

{

      // CSDATARANGE

     {

sizeof (CS_DATARANGE_VIDEO), // Flags

0,

SAMPLESIZE, // Replace with maximum Sample Size that the JPG data would take for the current resolution and image quality

0, // Reserved

STATIC_CSDATAFORMAT_TYPE_VIDEO,

MEDIASUBTYPE_IJPG,

STATIC_CSDATAFORMAT_SPECIFIER_VIDEOINFO

},

TRUE, // BOOL, bFixedSizeSamples (all samples same size?)

TRUE, // BOOL, bTemporalCompression (all I frames?)

CS_VIDEOSTREAM_CAPTURE, // StreamDescriptionFlags (CS_VIDEO_DESC_*)

0, // MemoryAllocationFlags (CS_VIDEO_ALLOC_*)

// _CS_VIDEO_STREAM_CONFIG_CAPS

{

// Omitted for this sample. Please refer to adapterprops.h in the sample null camera driver for ways to fill in this structure

………

………

},

// CS_VIDEOINFOHEADER

{

0,0,DX,DY, // RECT rcSource;

0,0,DX,DY, // RECT rcTarget;

BITRATE, // DWORD dwBitRate;

0L, // DWORD dwBitErrorRate;

REFTIME_15FPS, // REFERENCE_TIME AvgTimePerFrame;

sizeof (CS_BITMAPINFOHEADER), // DWORD biSize;

DX, // LONG biWidth;

DY, // LONG biHeight;

3, // WORD biPlanes;

DBITCOUNT, // WORD biBitCount;

FOURCC_IJPG | BI_SRCPREROTATE, // DWORD biCompression;

SAMPLESIZE, // DWORD biSizeImage;

0, // LONG biXPelsPerMeter;

0, // LONG biYPelsPerMeter;

0, // DWORD biClrUsed;

0, // DWORD biClrImportant;

0, 0, 0 // DWORD dwBitMasks[3]

}

};

Step 2: Add this format to the list of formats of Still pin

In sample null camera driver, you would modify CCameraDevice::Initialize() in cameradevice.cpp to have

    m_PinVideoFormat[STILL].categoryGUID = PINNAME_VIDEO_STILL;

    m_PinVideoFormat[STILL].ulAvailFormats = 1;

    m_PinVideoFormat[STILL].pCsDataRangeVideo[0] = &DCAM_StreamMode;

Step 3: Modify the BufferFill() function

Use biCompression field of the current format to determine whether MEDIASUBTYPE_IJPG format is selected. In BufferFill() you can have following check

UINT

BufferFill( PUCHAR pImage, PCS_VIDEOINFOHEADER pCsVideoInfoHdr, IMAGECOMMAND Command, bool FlipHorizontal, LPVOID lpParam )

{

    if( NULL == pCsVideoInfoHdr )

    {

        return -1;

    }

   

    DWORD biCompression = pCsVideoInfoHdr->bmiHeader.biCompression;

   

    if ( (FOURCC_IJPG == (biCompression & ~BI_SRCPREROTATE)) )

    {

      // Real drivers would add code here to handle JPEG encoding.

// The biSizeImage that is returned here should be the actual

// size of JPEG data that is written to the buffer.

        return biSizeImage;

    }

……….

……….

}

Step 4: Handle PROPSETID_VIDCAP_VIDEOCOMPRESSION

The default implementation(AdapterHandleCompressionRequests()) for PROPSETID_VIDCAP_VIDEOCOMPRESSION returns ERROR_INVALID_PARAMETER. Camera application on Windows Mobile 5.0 and later, uses this interface to set encoder quality. You would need to change this to return ERROR_SUCCESS after correctly handling various properties in this property set. Failure to do this would result in camera application raising an exception at initialization.

 

Camera diagram.JPG

Comments

  • Anonymous
    April 01, 2007
    Hi I m trying to capture camera buffer using Dshow, but i m not able find any information on that Can u help for that??? i m getting information about writting buffer to file, but i want camera buffer, as i need to send it over network. please help me , Thanks.   Test

  • Anonymous
    April 16, 2007
    Hello Saad, Why has the Imaging API been not yet updated/modified to accept YCbCr color format? As this will give an extra option for users developing Imaging encoders/decoders using MS provided ImagingAPI. You mention that, platforms with optimized JPEG encoders accepting YCbCr format at the input should perform this encoding inside the camera driver and provide the encoded data to Image Sink filter. Yes currently this is the only option! But on such platforms, applications like Image editing can not be accelerated for optimized encoding as there is no optimized Image encoder with RGB format (default encoder must be used). ImagingAPI accepting YCbCr will solve this issue. Regards, Sandeep Shinde

  • Anonymous
    April 20, 2007
    vishaljogi, What platform/version you are using. Is it based on Windows CE or Windows Mobile? Camera API was introduced in WM 5.0 and then shipped with WM 6.0 as well as Windows CE 6.0. http://msdn2.microsoft.com/en-us/library/aa451185.aspx should give you some information. -Saad

  • Anonymous
    April 20, 2007
    Sandeep, Thanks for you comments. You can still plug-in your optimized YUV encoder as an Imaging Plugin. You will have to do an RGB-YUV conversion in your plugin before feeding the data to your encoder. This way any editing application can make use of your optimized encoder. I know the concerns about the redundant RGB->YUV conversion but we need to live with that. -Saad

  • Anonymous
    May 07, 2007
    Thank you for the response. The conversion of course takes lot of memory (as still images are very big these days around 3MP). Could you please write a blog about the changes in camera driver structure (if any) from WM6.0 to Windows CE 6.0. As far as I know, the driver existing on WM6.0 will work with WinCE6.0 without any changes. Is this assumption correct. Regards, Sandeep

  • Anonymous
    May 07, 2007
    Saad, Thank you for the response. The conversion of course takes lot of memory (as still images are very big these days around 3MP). Could you please write a blog about the changes in camera driver structure (if any) from WM6.0 to Windows CE 6.0. As far as I know, the driver existing on WM6.0 will work with WinCE6.0 without any changes. Is this assumption correct. Regards, Sandeep

  • Anonymous
    August 23, 2007
    Hi Saad, I'm developing camera driver on WinCE5.0 (not Magneto or XBow). I expect to make the camera driver to support DShow framework. As I know, streaming architecture for cameras were added since WM5.0 or later. I found nothing for that in PB5.0(WinCE5.0) help document. So my question is, could I develop a DShow camera driver on "WinCE5.0"? If yes, how to do? Regards, Alec

  • Anonymous
    February 11, 2008
    Hi Saad, I am writing a DirestShow Camera application. I am loading the camera driver using the following code. varCamName = _T ("CAM1:"); hr = PropBag.Write( L"VCapName", &varCamName ); hr = pPropertyBag->Load( &PropBag, NULL); It starts well for the first time, but after destroying the graph, if the capture graph is started for the second time it gives the following error.. //hr fails with -2147024841 (ERROR_DEV_NOT_EXIST) What could be the reason ? Can u give some solution ? Thanks in advance, Sitharth.

  • Anonymous
    December 02, 2008
    Hello Saad, Can you please tell me how should I register a OEM developed Imaging encoder plug-in (IImageEncoder) with the default applications. There is no help available for this anywhere. Thanks in advance for the help, Regards, Sandeep