Webcam control with WPF or how to create high framerate player with DirectShow by using InteropBitmap in WPF application

Did you ever see, that MediaElement “eats” about 30% of CPU while playing movie in WPF? Did you thought, that you  can display live camera capture in WPF with 60 fps full screen (I have really high resolution 1920x1200) and 2% of  CPU? You did not? Let’s see how it can be done. Today we’ll create simple WebCam player control that can show you live video capturing with high frame rate. In order to do it, we’ll use DirectShow, WPF and make them work together.

 

You, probably do not believe me. Let’s start. In order to build this application, we need to make DirectDraw working in C# managed code. We can use DirectShow.NET, but this time we’ll do it manually. Why? because I love to do things manually. So let’s understand what we need? Actually, not very much: one Sample Grabber (ISampleGrabber) and one Device input filter(IBaseFilter). Both we should connvert with Graph Builder (IGraphBuilder) and point to some grabber implementation (ISampleGrabberCB). Also, we do not want DirectShow to render video for use, thus we’ll send it’s default Video Window (IVideoWindow) to null with no AutoShow and then run the controller (IMediaControl). Do you tired enough to lost me? Let’s see the code. One Filter graph with one Device Filter and one Sample grabber.

graph = Activator.CreateInstance(Type.GetTypeFromCLSID(FilterGraph)) as IGraphBuilder;
sourceObject = FilterInfo.CreateFilter(deviceMoniker);

grabber = Activator.CreateInstance(Type.GetTypeFromCLSID(SampleGrabber)) as ISampleGrabber;
grabberObject = grabber as IBaseFilter;

graph.AddFilter(sourceObject, "source");
graph.AddFilter(grabberObject, "grabber");

Set media type for our grabber

using (AMMediaType mediaType = new AMMediaType())
                {
                    mediaType.MajorType = MediaTypes.Video;
                    mediaType.SubType = MediaSubTypes.RGB32;
                    grabber.SetMediaType(mediaType);

And then connect device filter to out pin and grabber to in pin. Then get capabilities of video received (thiss stuff come from your web camera manufacturer)

if (graph.Connect(sourceObject.GetPin(PinDirection.Output, 0), grabberObject.GetPin(PinDirection.Input, 0)) >= 0)
                    {
                        if (grabber.GetConnectedMediaType(mediaType) == 0)
                        {
                            VideoInfoHeader header = (VideoInfoHeader)Marshal.PtrToStructure(mediaType.FormatPtr, typeof(VideoInfoHeader));
                            capGrabber.Width = header.BmiHeader.Width;
                            capGrabber.Height = header.BmiHeader.Height;
                        }
                    }

Out pin to grabber without buffering and callback to grabber object (this one will get all images from our source).

graph.Render(grabberObject.GetPin(PinDirection.Output, 0));
grabber.SetBufferSamples(false);
grabber.SetOneShot(false);
grabber.SetCallback(capGrabber, 1);

Dump output window

IVideoWindow wnd = (IVideoWindow)graph;
wnd.put_AutoShow(false);
wnd = null;

And run the controller

control = (IMediaControl)graph;
control.Run();

We done. Now our video is captured and can be accessed from BufferCB method of ISampleGrabberCB. Next step is to do WPF related stuff

First of all, we’ll use InteropBitmap. This one will provide us with real performance bust. So, one our DirectShow graph is ready and we know result image capabilities, we can create memory section and map it in order to provide ISampleGrabberCB with place to put images. This will be always the same pointer, so all we have to do is to .Invalidate interop image.

if (capGrabber.Width != default(int) && capGrabber.Height != default(int))
                {

                    uint pcount = (uint)(capGrabber.Width * capGrabber.Height * PixelFormats.Bgr32.BitsPerPixel / 8);
                    section = CreateFileMapping(new IntPtr(-1), IntPtr.Zero, 0x04, 0, pcount, null);
                    map = MapViewOfFile(section, 0xF001F, 0, 0, pcount);
                    BitmapSource = System.Windows.Interop.Imaging.CreateBitmapSourceFromMemorySection(section, capGrabber.Width, capGrabber.Height, PixelFormats.Bgr32,
                        capGrabber.Width * PixelFormats.Bgr32.BitsPerPixel / 8, 0) as InteropBitmap;
                    capGrabber.Map = map;
                    if (OnNewBitmapReady != null)
                        OnNewBitmapReady(this, null);
                }

Now in capGrabber (ISampleGrabberCB) we’ll copy buffer, comes from our webcam to the mapped location for WPF usage

public int BufferCB(double sampleTime, IntPtr buffer, int bufferLen)
        {
            if (Map != IntPtr.Zero)
            {
                CopyMemory(Map, buffer, bufferLen);
                OnNewFrameArrived();
            }
            return 0;
        }

All we have to do is to call InteropBitmap.Invalidate() each frame to reread the image bytes from the mapped section.

if (BitmapSource != null)
                    {
                        BitmapSource.Invalidate();

How do display all this stuff? Simple – subclass from Image and set it’s Source property with the interop bitmap.

public class CapPlayer : Image,IDisposable
    {
…

void _device_OnNewBitmapReady(object sender, EventArgs e)
        {
            this.Source = _device.BitmapSource;
        }

Now, the usage from XAML is really simple

<l:CapPlayer x:Name="player"/>

We done :) As always, download full source code for this article

…and be good people and don’t tell anymore, that WPF performance in terms of imaging is sucks :)

P.S. small ‘r’ if you have more, then one WebCam connected. Inside CapDevice class there is member, public static FilterInfo[] DeviceMonikes, that provides you with all DirectShow devices installed. So, the only thing you should do in order to change the device is to set deviceMoniker = DeviceMonikes[0].MonikerString; with the moniker of your device. This sample works with first one.