Partager via

Multithreaded UI: HostVisual

  • Article

Background: The WPF Threading Model

In general, objects in WPF can only be accessed from the thread that created them. Sometimes this restriction is confused with the UI thread, but this is not true, and it is perfectly fine for objects to live on other threads. But it is not generally possible to create an object on one thread, and access it from another. In almost all cases this will result in an InvalidOperationException, stating that “The calling thread cannot access this object because a different thread owns it.”

Freezables

Of course, there are exceptions to this restriction. A familiar one is the Freezable class. Freezable objects can be frozen, at which point they become read-only and we lift the single-thread restriction. Examples of frozen Freezables include the standard brushes, available from the Brushes class. These brushes can be used on any thread at any time.

This is incredibly useful, and used for all of the graphics primitive resources (pens, brushes, transforms, etc). You can even derive your own types from Freezable and play by the same rules.

Separate Windows

Unfortunately, that read-only restriction can be a real problem. There are many scenarios where we want to run separate pieces of the UI on separate threads. If these pieces of UI are independent from each other, you can host them in separate windows, and run the windows on separate threads. This can be a reasonable solution for some scenarios, especially scenarios where the separate threads can be run in independent top-level windows. The big restriction for this approach is that the graphics from one cannot be composited with the graphics of the other. So while you could use a child window for the other thread’s UI, the system will just render one on top of the other. You cannot have transparency, you cannot use one as a brush for 3D content, you can’t draw over it, etc.

HostVisual

If your scenario doesn’t require interactivity (meaning input), then there is another option that WPF provides: HostVisual. This option leverages the powerful composition engine in WPF that is already capable of aggregating the rendering primitives from multiple threads into one scene. The element tree owned by the worker thread is rendered into its own composition target (called VisualTarget), and the results are composed into the HostVisual owned by the UI thread.

Issue #1: Hosting a Visual in XAML

The first problem to solve is that the HostVisual class derives from Visual. I can't use an existing panel, such as Border, to host this visual. Border derives from Decorator, which is the standard base class for panels that have a single child. Unfortunately, the child is strongly typed to be a UIElement. I have to use a HostVisual, which does not derive from UIElement. There is no built-in way that I know of to place a Visual as a child of one of the standard elements (such as Border, Grid, Canvas, etc). So we make our own:

 

    [ContentProperty("Child")]

    public class VisualWrapper : FrameworkElement

    {

        public Visual Child

        {

            get

            {

                return _child;

            }

            set

            {

                if (_child != null)

                {

                    RemoveVisualChild(_child);

                }

                _child = value;

                if (_child != null)

                {

                    AddVisualChild(_child);

                }

            }

        }

        protected override Visual GetVisualChild(int index)

        {

            if (_child != null && index == 0)

            {

                return _child;

            }

            else

            {

                throw new ArgumentOutOfRangeException("index");

            }

        }

        protected override int VisualChildrenCount

        {

            get

            {

                return _child != null ? 1 : 0;

            }

        }

        private Visual _child;

    }

 

 

Issue #2: Layout and the Loaded event

WPF provides a very convenient event called “Loaded”. This event basically signals when an element has been fully initialized, measured, arranged, rendered, and plugged into a presentation source (such as a window). Many elements use this event, including the MediaElement, but sadly this event is not raised for element trees that are not plugged into a presentation source, and displaying an element tree through the HostVisual/VisualTarget doesn’t count. So work around this, we make our own presentation source and use it to root the element tree that the worker thread will own. This immediately leads into another problem: layout is suspended on all elements until resumed by a presentation source. Unfortunately the official mechanism to do this is internal, so the best we can do is to explicitly measure and arrange the root element. Thus we have our VisualTargetPresentationSource class:

    public class VisualTargetPresentationSource : PresentationSource

    {

        public VisualTargetPresentationSource(HostVisual hostVisual)

        {

            _visualTarget = new VisualTarget(hostVisual);

        }

        public override Visual RootVisual

        {

            get

            {

                return _visualTarget.RootVisual;

            }

            set

            {

                Visual oldRoot = _visualTarget.RootVisual;

                // Set the root visual of the VisualTarget. This visual will

                // now be used to visually compose the scene.

                _visualTarget.RootVisual = value;

                // Tell the PresentationSource that the root visual has

                // changed. This kicks off a bunch of stuff like the

                // Loaded event.

                RootChanged(oldRoot, value);

                // Kickoff layout...

                UIElement rootElement = value as UIElement;

                if (rootElement != null)

                {

                    rootElement.Measure(new Size(Double.PositiveInfinity,

                                                 Double.PositiveInfinity));

                    rootElement.Arrange(new Rect(rootElement.DesiredSize));

                }

            }

        }

        protected override CompositionTarget GetCompositionTargetCore()

        {

        return _visualTarget;

        }

        public override bool IsDisposed

        {

            get

            {

                // We don't support disposing this object.

                return false;

            }

        }

        private VisualTarget _visualTarget;

    }

 

 

Background Threads

It is easy to make threads in C#. One trick to be aware of is that you must mark the thread as being a “background” thread, otherwise the application will keep running as long as those threads are alive. Also remember that parts of WPF require that its threads to be initialized for COM’s “Single Threaded Apartment”. All of this is easy enough to do, and you’ll see this code later on:

Thread thread = new Thread(/*…*/);

thread.ApartmentState = ApartmentState.STA;

thread.IsBackground = true;thread.Start(/*…*/);

 

 

The Demo

The demo will be a grid showing 3 movies, each rendered on a different background thread. The demo is very simple, but hopefully gets the salient points across. It is certainly possible to communicate from the UI thread to the worker threads, and given that all threads have their own dispatchers this is actually really easy, but this demo does not do that. Interested readers should consult the documentation for Dispatcher.BeginInvoke.

The XAML

The XAML simply defines a grid with 3 columns and puts a VisualWrapper in each column. This is where we will be putting the HostVisual from code. It would be cool to build a class that automatically displays its content on another thread, but we would have to change the parser to support switching threads since the thread an object is created on is the thread that owns that object. Anyways, the simple XAML is:

 

<Window x:Class="VisualTargetDemo.Window1"

    xmlns="schemas.microsoft.com/winfx/2006/xaml/presentation"

    xmlns:x="schemas.microsoft.com/winfx/2006/xaml"

    xmlns:local="clr-namespace:VisualTargetDemo"

    Title="VisualTargetDemo"

    SizeToContent="WidthAndHeight"

    Loaded="OnLoaded"

    >

  <Grid>

    <Grid.ColumnDefinitions>

      <ColumnDefinition Width="*"/>

      <ColumnDefinition Width="*"/>

      <ColumnDefinition Width="*"/>

    </Grid.ColumnDefinitions>

    <local:VisualWrapper Grid.Column="0"

                         Width="200" Height="100" x:Name="Player1"/>

    <local:VisualWrapper Grid.Column="1"

                         Width="200" Height="100" x:Name="Player2"/>

    <local:VisualWrapper Grid.Column="2"

                         Width="200" Height="100" x:Name="Player3"/>

  </Grid>

</Window>

 

 

The Code

The code is also pretty simple. For each of the 3 players, it creates a HostVisual on the UI thread, then spins up a background thread, creates a MediaElement, puts it inside a VisualTarget (which points back to the HostVisual), and puts it all inside our hacky VisualTargetPresentationSource.

    public partial class Window1 : System.Windows.Window

    {

        public Window1()

        {

            InitializeComponent();

        }

        private void OnLoaded(object sender, RoutedEventArgs e)

        {

            Player1.Child = CreateMediaElementOnWorkerThread();

            Player2.Child = CreateMediaElementOnWorkerThread();

        Player3.Child = CreateMediaElementOnWorkerThread();

        }

        private HostVisual CreateMediaElementOnWorkerThread()

        {

            // Create the HostVisual that will "contain" the VisualTarget

            // on the worker thread.

      HostVisual hostVisual = new HostVisual();

            // Spin up a worker thread, and pass it the HostVisual that it

            // should be part of.

            Thread thread = new Thread(new ParameterizedThreadStart(MediaWorkerThread));

         thread.ApartmentState = ApartmentState.STA;

            thread.IsBackground = true;

            thread.Start(hostVisual);

            // Wait for the worker thread to spin up and create the VisualTarget.

            s_event.WaitOne();

            return hostVisual;

        }

        private FrameworkElement CreateMediaElement()

        {

            // Create a MediaElement, and give it some video content.

            MediaElement mediaElement = new MediaElement();

            mediaElement.BeginInit();

            mediaElement.Source = new Uri("download.microsoft.com/download/2/C/4/2C433161-F56C-4BAB-BBC5-B8C6F240AFCC/SL_0410_448x256_300kb_2passCBR.wmv?amp;clcid=0x409");

            mediaElement.Width = 200;

            mediaElement.Height = 100;

            mediaElement.EndInit();

            return mediaElement;

        }

        private void MediaWorkerThread(object arg)

        {

            // Create the VisualTargetPresentationSource and then signal the

            // calling thread, so that it can continue without waiting for us.

            HostVisual hostVisual = (HostVisual)arg;

            VisualTargetPresentationSource visualTargetPS = new VisualTargetPresentationSource(hostVisual);

            s_event.Set();

            // Create a MediaElement and use it as the root visual for the

            // VisualTarget.

            visualTargetPS.RootVisual = CreateMediaElement();

            // Run a dispatcher for this worker thread. This is the central

            // processing loop for WPF.

            System.Windows.Threading.Dispatcher.Run();

        }

        private static AutoResetEvent s_event = new AutoResetEvent(false);

    }

 

 

Summary

In this demo I’ve shown how to use the HostVisual and VisualTarget classes to compose pieces of UI from different threads. There are some limitations: namely that the UI owned by the worker threads do not receive input events. There were also some annoyances we had to work around along the way, but those proved to be fairly minimal.

The source code for this project is now included in my codeplex site: <www.codeplex.com/MicrosoftDwayneNeed>

Comments

  • Anonymous
    May 01, 2007
    Multithreading support in WPF has been much on my mind lately, especially with my MediaBarrage app. Dwaybe

  • Anonymous
    May 01, 2007
    Client-side multi-threading is something I do day-in, day-out, and I have to say that the whole treatment of thread-safety for data held/cached on the client is rather poor. The only thing we have today (and I mean since .NET 1.0, and apparently going into 3.5) is the SynchronizationAttribute placed on classes that inherit from ContextBoundObject. Of course, this comes with a huge performance penalty. Is there something that WPF does that doesn't appear at the BCL level? Also, on the whole issue of STA - there are certain memory leaks that can only be handled by using MTA. I ran into this a while ago and as far as I know, the internals affecting this haven't been changed: http://udidahan.weblogs.us/2004/07/27/memory-leak-mopped-up/ I'm hoping you'll tell me I'm wrong :)

  • Anonymous
    June 10, 2007
    I traveled most of the week so did not blog ... from the blogs, I read here are some of my 'faves' for

  • Anonymous
    September 26, 2007
    There was a query on the MSDN forums for WPF the other day that asked about leveraging multiple processors

  • Anonymous
    March 02, 2008
    Microsoft DPE : &quot;WPF is ever best super performance oriented technology for creating hyper multi

  • Anonymous
    May 04, 2008
    The comment has been removed

  • Anonymous
    June 22, 2008
    The comment has been removed

  • Anonymous
    December 14, 2008
    Basically, window app UI, either WPF or traditional WinForm, is single threaded, which means only one

  • Anonymous
    January 01, 2009
    [This blog was migrated. You will not be able to comment here. The new URL of this post is http://khason

  • Anonymous
    January 01, 2009
    [This blog was migrated. You will not be able to comment here. The new URL of this post is http://khason

  • Anonymous
    April 19, 2010
    How about binding? Can we bind property of one element to another element's property running in different thread?

  • Anonymous
    May 26, 2010
    Multithreaded UI, this is what I have been looking for days. I just try your example, it is working. However, I can't get it working with other controls. If I replace the MediaElement with other controls, like a WebBrowser, the window is blank. Do you have any idea how I can use HostVisual with WebBrowser? Thank you in advance.

  • Anonymous
    August 03, 2010
    Hi I have been looking at your post and it looks great, however I had one slight problem with the following type: VisualTargetPresentationSource I get a complile error saying type is unknown - the problem is I do not know what assembly this type is found in? Any hints? Is it in the Visual Studio 2010 SDK?   Any help greatly appreciated! Cheers,

  • Anonymous
    August 30, 2010
    I have been looking for something like this, great help. However, It seems you set this up as a clickOnce deployment or something. I downloaded the source, but when I try to run it I get an InvalidDeploymentException, Application Identity not set exception. Any ideas on how to get around that?

  • Anonymous
    September 07, 2010
    Its great article. but I have a question. How do I make display in a background thread a constantly changing random set of points instead of the video? Could you give an example? lexeich (AT) nxt (.) ru. sorry for my english

  • Anonymous
    December 02, 2010
    It is great work, still it is written in 2007 means consumes .NET 3.0 features. My concern is I want to have interactions with the UI elements hosted inside the host visual (Let say I want to control the video by clicking on top of video element). I tried so far but still I couldn't achieve. Is there any workarounds with the newer versions of .NET to handle multi threaded UI? Is there any possibility to route the events towards the hosted element?

  • Anonymous
    April 05, 2012
    It is great work, But how about scenario that requires interactivity?

  • Anonymous
    February 28, 2013
    /// <summary>    /// welcome window    /// </summary>    public partial class Welcome : Window    {        public Welcome()        {            InitializeComponent();            this.Loaded += Welcome_Loaded;        }        private void Welcome_Loaded(object sender, RoutedEventArgs e)        {            Thread asyncLoadMian = new Thread(() =>            {                MainWindow mainWindow = new MainWindow();                mainWindow.Show();                App.Current.Dispatcher.BeginInvoke(new Action(() => { this.Hide(); }));                System.Windows.Threading.Dispatcher.Run();            });            asyncLoadMian.IsBackground = false;            asyncLoadMian.SetApartmentState(ApartmentState.STA);            asyncLoadMian.Start();        } } This is my application.  If use the visualHost  is throw the expection.  

  • Anonymous
    December 12, 2014
    Does this only work with media elements?

  • Anonymous
    January 08, 2015
    Great article, thanks for sharing! I just wonder if this can be used along with PRISM. Do you have any suggestion or idea to share? @Alex: technically, this can work for any FramewokElement you return from following function private FrameworkElement CreateMediaElement()

  • Anonymous
    August 05, 2016
    The comment has been removed

  • Anonymous
    August 16, 2016
    How can we use this approach with a custom complex control, which is created in XAML, and cannot be created in the code behind? This custom control contains an ItemsCollection with a complex data template and takes a long time to be displayed on the screen. If I try to set this control as the RootVisual of the visual target I am receiving an exception that the control is created in another thread.Thanks

  • Anonymous
    August 31, 2016
    I use my own User Control like Host Visual. Is it possible make Host Visual active for user's inputs? Actually no event is handled.

  • Anonymous
    May 18, 2017
    Hello and thank you for a life saving post. I have implemented this nicely with some streaming charts.Previously these charts auto-sized to fit their container. Now that they are inside HostVisual, I've had to fix their height and width. Is it possible for HostVisual to resize with the app?