https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/Visual-Studio-Toolbox-Debugger-Canvas

Prototyped Features

All Code Simultaneously on a Single Zoomable Surface

View Code based on More than just Files and Directories

Fluid Semantic Zoom

Automatic Layout and Edge Routing

Groups

Annotations

Ambient Information

Control Structure and References

Off-Screen References

Search Results

Execution Trace Playback

Continuing Research

Better Language Support

Visual Designers

Inline Editing and Real-Time Updates

Better Layout and Edge Routing

Semantic Annotations and “Hand-Drawings”

Task-Specific and Filtered Views

Minimap(s) and Extra Views

Navigation and Key-Bindings

General User Experience

• Search results are currently all shown individually.  When zoomed in, the results that are off the screen are semi-transparent and when they overlap they combine to become more opaque.  But search results can also overlap when zooming out since content gets closer and closer together, and translucency is not really appropriate in that case.  Instead, multiple overlapping search results should coalesce into a single ‘blob’ that shows an aggregate of the results.  The coalescence should happen dynamically and fluidly as the user zooms in and out instead of simply snapping from one representation to another.
• Groups look nice when zoomed out large enough to see the entire group; you can see their entire border and label at the top.  But when zooming into something within the group, the border and label are often off the edge of the screen and you can no longer tell which group(s) you are in.  We should use some visual effects to keep the group borders/labels constrained to the edge of the screen (just like search results are).  If there are several groups surrounding the current view (e.g. a deeply nested hierarchy) then we should attempt to combine their visual representation into a single border/label that aggregates their purpose.
• We currently use simple pastel-colored highlighting to represent things like code coverage, but when it comes to things like error-prone code (that is, code which has had multiple bugs filed against it) or high-churn code (code which has had multiple checkins or checkins from multiple people) or cloned-code etc., it may be more appropriate to use game-like graphics like green wafts of “stink” or tiny “bugs” crawling around the code.  This would be more than just fun, it may also be a sort of social conditioning to help developers instinctively avoid writing “bad code” for fear of their section of code looking like garbage.  Likewise, perhaps very good code should have a glow or other pleasing affect applied to it in order for the author to “show off” his expert coding skills whenever someone zooms out to see the entire codebase at large.
• Some users have expressed that zooming in and out to see details is overwhelming and some even report motion sickness.  Perhaps a good option for these users is a way to have a “magnifying glass” or “pop-up editors” so that the user can stay mostly zoomed out all the time but enlarge specific areas to see details or edit code.  The challenge then becomes how to deal with the pop-ups if they last for any significant amount of time.  For example, a section of the canvas is magnified or “popped-up” and you pan the canvas beneath it, should the popups move as well?  Or should they stationary on the screen?  And when a code editor is “popped-up”, should you be able to scroll up and down within it?  Perhaps even be able to pan side-to-side in it?  Should a pop-up be able to show more than one item at a time?  These are the types of questions that we have yet to explore and research, and our primary goal is to find whatever is optimal for software developers and software development instead of solving the problem for ZUIs in general (but of course we will be pleased if our research benefits other ZUIs as well).

Ease of Getting Started

New and Unexplored Research Areas

Cross-Solution, Distributed System, and Dependency Layering

Team Activity and Ownership

Data Flow Analysis and Visualization

Runtime Heap & Object Graph Visualization

Comparing “Diffs” of Code and Changes Over Time

Code Canvas: Shows source code and relationships among source code on a zoomable canvas in a spatially stable and meaningful way. It allows code to be edited directly on the canvas instead of in separate tabbed editors like traditional IDEs. Semantic zoom is used to zoom out to view the entire system and its architecture in a smooth “deep zoom” kind of way. Information overlays like references, code coverage, and execution traces are displayed directly within the canvas instead of a separate diagram or window. The position of each item on the canvas has meaning in the context of the overall software diagram and is not placed arbitrarily. Each item on the canvas has one and only one location and is never duplicated (although it can be moved). Development teams share and collaborate over a single master canvas in the same way they agree on the locations of files in their source control directory tree. The lifetime of the canvas is mostly expected to be permanent and become a valued asset of the project. Spatial memory is a primary focus.

Code Bubbles: Replaces the entire IDE with a single canvas. It contains source code fragments but also can contain tool windows, property grids, tear-off queries, work items, email, etc. Some windows (like menus or property grids) may appear just temporarily in a spatial radius around the “current item” instead of always being visible on the canvas. The position of each item on the canvas is meant to maximize organization or screen real-estate for the current task or environment. Items (including source code fragments) can be duplicated and appear in multiple places at once. Development teams may give copies of canvases to team-mates or new hires but they are usually developer-specific. The lifetime of the canvas is usually temporary and/or task-specific. Spatial memory may be relevant but is not a primary focus.

Debugger Canvas: Shows source code fragments on a canvas for each step of a program being debugged, with arrows indicating where one method stepped into another during that debugging session. It also contains non-code items such as “locals” windows on the canvas and allows debugging-specific operations such as taking a snapshot of the local variables. The position of each item on the canvas is related directly to the order (or thread) in which it was called during a debugging session. The user has the option of duplicating source code fragments on the canvas each time the code is stepped into, or they can choose to reuse the same fragment over and over if it is called more than once. Developers may send static screenshots or copies of the canvas to teammates but otherwise canvases are not shared. The lifetime of the canvas is only meant to last for a single debugging session. Spatial memory is only a focus in that it is meant to be consistent with the order of execution of the program.

- Easily turn Debugger Canvas on or off, even in the middle of a debug session.
- Quickly navigate up and down the call stack in the canvas using the keyboard or the mouse.
- See Multiple threads together on the canvas, each thread identified by its color.
- See recursive calls side by side.
- The option to edit code directly in the code bubbles is enabled by default.

This last "feature" is actually nothing new. You could always edit code directly in bubbles by checking the checkbox in Tools -> Options, but some users didn't know that option existed so we frequently heard the feedback that you wished you could edit code directly in the bubbles. This time we just simply enable that feature by default if you're installing Debugger Canvas for the first time.

To install this updated version of Debugger Canvas you must uninstall the previous version of Debugger Canvas first. (Or install QFE 2581019 from http://connect.microsoft.com/VisualStudio/Downloads/DownloadDetails.aspx?DownloadID=38654 before installing the update.) This is because Visual Studio 2010 SP1 has a bug which prevents signed VSIXes from being updated. I will also post a sticky thread in the forum since I'm sure many of you will attempt to install version 1.1 on top of 1.0 without reading the instructions first. I would do the same thing! :)

You can read more about the new version and read the updated guide from the usual home page here:
http://msdn.microsoft.com/en-us/devlabs/debuggercanvas

Or you can skip straight to the download and get the VSIX directly from here:
http://download.microsoft.com/download/5/A/7/5A7FB2E7-5720-4739-BDDE-28A684C5B291/Microsoft.DebuggerCanvas.vsix

Thanks to everyone for their continued support and we sincerely hope you enjoy Debugger Canvas version 1.1!

We’ve heard you loud and clear that you wish Debugger Canvas wasn’t Ultimate-only.  We would love to make Debugger Canvas available to a wider audience, but as Soma says on his blog, Debugger Canvas is built on top of Visual Studio Ultimate so that we could re-use the underlying technology for the Dependency Diagrams to identify and display the right fragments on the canvas.  This re-use is what made it possible for us to release an out-of-band power tool in an efficient manner on top of Visual Studio 2010.  It's unlikely that we'll be able to make this version of the power tool available in Pro, but please let us know if you'd like to see this kind of user experience built directly "in the box" as part of Visual Studio.

In the mean time, if you don’t have Visual Studio Ultimate then you still have two options to try Debugger Canvas for free:

1) Install the free Visual Studio Ultimate Trial, or
2) Install the free Virtual Machine containing Visual Studio Ultimate.

The second option is probably the most convenient since Visual Studio Ultimate is already pre-installed on the machine.  All you need to do is copy the Debugger Canvas VSIX into the virtual machine and install it.

Please send us your questions and feedback on the Debugger Canvas DevLabs Forum.  Thanks!

To show you an example, I’m going to start with the ZoomableApplication2 from the last post, but I’m going to modify it slightly by setting the Padding to 0 in the ItemContainerStyle and adding Stroke="Black" to the <Rectangle>s and <Ellipse>s (to give them a black outline).  If you haven’t read that post yet, you might want to do so now.

Next, I’m going to set ApplyTransform="False" and modify the Width and Height setters in the <ListBox.ItemContainerStyle> to multiply them by the Scale of the ZoomableCanvas, like this:

<Setter Property="Width">
    <Setter.Value>
        <MultiBinding Converter="{x:Static ArithmeticConverter.Default}"
                      ConverterParameter="*">
            <Binding Path="width"/>
            <Binding Path="Scale"
                     RelativeSource="{RelativeSource
                                      AncestorType=ZoomableCanvas}"/>
        </MultiBinding>
    </Setter.Value>
</Setter>

<Setter Property="Height">
    <Setter.Value>
        <MultiBinding Converter="{x:Static ArithmeticConverter.Default}"
                      ConverterParameter="*">
            <Binding Path="height"/>
            <Binding Path="Scale"
                     RelativeSource="{RelativeSource
                                      AncestorType=ZoomableCanvas}"/>
        </MultiBinding>
    </Setter.Value>
</Setter>

Notice I’m using my trusty ArithmeticConverter from Presentation.More to do the multiplication (since XAML doesn’t support math directly).  This has the effect of simply resizing the shapes instead of scaling them, but it does nothing to the StrokeThickness or the <TextBlock> inside.  Here are the results side-by-side, with ApplyTransform="True" on the left and ApplyTransform="False" on the right:

(I’ve embedded the sample as an XBAP in an <iframe> above this line, so you should see a live sample and be able to interact with it if you’re using Internet Explorer and have .NET 4.0 installed.)

Notice as you zoom out that it becomes hard to see the black outline around the shapes on the left, but the shapes on the right always maintain the same thickness.  The same is true for the curvy paths.  Also notice how the text labels on the left quickly become too small to see, but the labels on the right are always visible until clipped.

Another situation when to set ApplyTransform="False" is when you want certain items to scale at a different rate than the rest of the canvas.  For example, say you want to be able to add flags or pushpins as landmarks on your canvas, but you don’t want them to become too small when you zoom all the way out.  In this case, you’ll still be applying a ScaleTransform to your items, but each item will have an individual transform instead of sharing one big one.  This is because the location of your landmarks will be changing at a different rate than the scale of the landmarks, so the scene as a whole does not transform uniformly.

To add an overlay with landmarks on top of the canvas, I’m just going to create a second <ZoomableCanvas> on top of the first one, but this time I’m going to use a bare-bones <ItemsControl> instead of a <ListBox>.  This is because the first <ListBox> already gives me scroll bars and keyboard navigation, and I don’t need my landmarks to be selectable.  I’ll place them on top of each other by changing my <DockPanel> into a <Grid> with two rows, and placing both controls in the same row:

<Grid>

    <Grid.RowDefinitions>
        <RowDefinition Height="Auto"/>
        <RowDefinition Height="*"/>
    </Grid.RowDefinitions>

    <Slider x:Name="MySlider" Grid.Row="0"
            Maximum="1000000" AutoToolTipPlacement="BottomRight"
            ValueChanged="MySlider_ValueChanged"/>

    <ListBox x:Name="MyListBox" Grid.Row="1">

        . . .

    </ListBox>

    <ItemsControl x:Name="MyLandmarks" Grid.Row="1"
                  VerticalAlignment="Top"
                  HorizontalAlignment="Left"
                  Width="{Binding ActualWidth}"
                  Height="{Binding ActualHeight}"
                  Margin="2">

        . . .
        
    </ItemsControl>

</Grid>

The first thing you might notice is that I’m setting Width="{Binding ActualWidth}" and the same for Height.  But what am I binding to?  In the code behind, I’ve assigned the DataContext to the first ZoomableCanvas so that I can access it via {Binding}s.

private void ZoomableCanvas_Loaded(object sender, RoutedEventArgs e)
{
    // Store the canvas in a local variable since x:Name doesn't work.
    MyCanvas = (ZoomableCanvas)sender;

    // Set the canvas as the DataContext so our overlays can bind to it.
    DataContext = MyCanvas;
}

My overlay <ItemsControl> needs to match the Width and Height of the base canvas explicitly because the size keeps changing as the scroll bars appear and disappear.  If I didn’t make the overlay match then it would draw on top of the scroll bars which would look pretty weird.  Another advantage of setting DataContext = MyCanvas is that I can also match the Scale and Offset in the same way:

<ItemsControl.ItemsPanel>
    <ItemsPanelTemplate>
        <ZoomableCanvas Scale="{Binding Scale}"
                        Offset="{Binding Offset}"
                        ApplyTransform="False"
                        ClipToBounds="True"/>
    </ItemsPanelTemplate>
</ItemsControl.ItemsPanel>

The ItemContainerStyle for our landmarks looks similar to the one for our main items, except this time instead of binding the Width and Height to a factor of the Scale, we’ll set the RenderTransform to a ScaleTransform instead:

<ItemsControl.ItemContainerStyle>
    <Style TargetType="ContentPresenter">
        
        <Setter Property="Canvas.Top" Value="{Binding top}"/>
        <Setter Property="Canvas.Left" Value="{Binding left}"/>
        
        <Setter Property="RenderTransform">
            <Setter.Value>
                <ScaleTransform ScaleX="{Binding ScaleY,
                                RelativeSource={RelativeSource Self}}">
                    <ScaleTransform.ScaleY>
                        
<MultiBinding Converter="{x:Static ArithmeticConverter.Default}"
              ConverterParameter="^">
    <Binding Path="Scale"
             RelativeSource="{RelativeSource
                              AncestorType=ZoomableCanvas}"/>
    <Binding Source=".333"/>
</MultiBinding>
                        
                    </ScaleTransform.ScaleY>
                </ScaleTransform>
            </Setter.Value>
        </Setter>
    </Style>
</ItemsControl.ItemContainerStyle>

(Sorry about the weird formatting - the blog width is really small.)

We’re setting the landmark scale to the cube-root of the main scale (Math.Pow(Scale, .333)).  We are doing this by passing "^" as the ConverterParameter to our ArithmeticConverter as before.  This results in the scale of our landmarks changing much slower than the rest of the canvas, so as you zoom out they stay larger longer, and as you zoom in they stay smaller longer.  Of course you’re welcome to tweak the formula and try completely different equations instead.

For the landmark visual itself, I have a pretty red pushpin that was given to me by Lutz Gerhard.  In fact, he was the one who originally told me about “power-law scaling” in the first place, so a lot of the credit goes to him.

<ItemsControl.ItemTemplate>
    <DataTemplate>
        <Image Source="Pushpin.png" Margin="-8,-61,0,0"/>
    </DataTemplate>
</ItemsControl.ItemTemplate>

The “tip” of the pushpin (the point at which it appears to punch through the surface) is about 8 pixels over and 61 pixels down in the image, so I’m setting the Margin to center it on that point.  Now I’ll simply add a new landmark whenever you double-click:

protected override void OnMouseDoubleClick(MouseButtonEventArgs e)
{
    var position = MyCanvas.MousePosition;
    MyLandmarks.Items.Add(new { top = position.Y, left = position.X });
}

And this is what you get!

You can still use the slider to add and remove items, and double-click to add pushpins.  Try adding 1,000,000 items and adding pushpins at each of the 4 corners.  Then you can zoom all the way out and get a rough idea of how big a million items really is!  You’ll also notice that you can double-click when you’ve zoomed and panned around, and the pushpin still ends up at the right location.  This is because we are using MyCanvas.MousePosition instead of e.GetPosition().  If we were to use the latter, then it would always give us the distance relative to the top-left corner of the control on the screen, so if you double-clicked at the top-left corner of the <ListBox> then you’d always get a position of (0,0) even if the canvas was scrolled all the way to the bottom-right!  ZoomableCanvas.MousePosition is a handy property that will always give you the coordinates of the mouse pointer in canvas coordinates, so you don’t have to do the computation to figure it out yourself.   ZoomableCanvas also has GetCanvasPoint(screenPoint) and GetVisualPoint(canvasPoint) if you want to do the same computations but with arbitrary points.

I hope this has given you enough information on how to use the different modes of ZoomableCanvas appropriately.  I’ve attached the source code for ZoomableApplication3 to this post, so hopefully you’ll be up and running with cool effects in no time.  And by using the Visual State Manager in Expression Blend to completely switch the appearance of your items based on the Scale, you’ll be well on your way to implementing semantic zoom.

ZoomableApplication3.zip

There are a few things to keep in mind with that original sample.  First, in order to get this UI virtualization, your data items were responsible for creating and disposing their own UIElements and visual trees directly from their code-behind, instead of the usual practice of defining <DataTemplate>s somewhere else in XAML.  Second, items had to be added to the VirtualCanvas directly, instead of being able to use data-binding with an ItemsControl like a <ListBox>.  Third, all data items had to be loaded and remain in RAM even when their respective UIElements are not.  50,000 classes or structs by themselves might not be a big deal, but if it takes over a millisecond to load each item then you’re talking about a very significant startup time.  Fourth, if the user zoomed all the way out then all of the UIElements were created at once which would cause the app to hang or simply crash altogether.

For the second post in the ZoomableCanvas series I’m going to recreate the original random shapes sample, but this time I’m going to use <DataTemplate>s and a <ListBox> so that we get things like selection, scrolling, and keyboard navigation for free.  I’m also going to show you how UI and data virtualization work, and how you can zoom all the way out and still have a responsive app even with 1,000,000 items.

I’m going to start by creating a brand new app called ZoomableApplication2 and adding a reference to the ZoomableCanvas.dll class library that I built for ZoomableApplication1.  Then I’m going to replace the default <Grid> with a <ListBox> and set the <ListBox.ItemsPanel> to a <ZoomableCanvas>, like this:

<Window x:Class="ZoomableApplication2.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525">
    <ListBox x:Name="MyListBox">
        <ListBox.ItemsPanel>
            <ItemsPanelTemplate>
                <ZoomableCanvas/>
            </ItemsPanelTemplate>
        </ListBox.ItemsPanel>
    </ListBox>
</Window>

Now lets add a bunch of items to the <ListBox>.  Each of our items will have a top, left, width, and height that places them in a grid-like pattern, kind of like the original VirtualCanvas sample.  We’ll also give them some random control points for the curves and random colors to make them look pretty.  And since we’re going to be using {Binding}s when we write our XAML, our data items don’t even need to be strongly typed. They can just be anonymous bags of properties:

public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();

        DataItems = new ObservableCollection<object>();
        MyListBox.ItemsSource = DataItems;

        PopulateDataItems(100);
    }

    private ObservableCollection<object> DataItems;

    private void PopulateDataItems(int newCount)
    {
        // Remove items if our new count is less than our current count.
        while (newCount < DataItems.Count)
        {
            DataItems.RemoveAt(DataItems.Count - 1);
        }
        // Add items if our new count is more than our current count.
        while (newCount > DataItems.Count)
        {
            // Give each item a deterministic seed.
            var i = DataItems.Count;
            var rand = new Random(i);
            var sqrt = (int)Math.Sqrt(i);

            // With a random width and height between 50 and 100.
            var width = rand.Next(50, 100);
            var height = rand.Next(50, 100);

            // And place it at a random position in a grid-like pattern.
            var top = Math.Min(sqrt, i - Math.Pow(sqrt, 2))
                      * 100 + rand.Next(100 - height);
            var left = Math.Min(sqrt, sqrt * 2 - (i - Math.Pow(sqrt, 2)))
                       * 100 + rand.Next(100 - width);

            // Randomly generate the outline of the item.
            var type = rand.Next(3);
            var data = type == 0 ? "ellipse" :
                       type == 1 ? "rectangle" :
                       String.Format("M{0},{1} C{2},{3} {4},{5} {6},{7}",
                           rand.NextDouble(),
                           rand.NextDouble(),
                           rand.NextDouble(),
                           rand.NextDouble(),
                           rand.NextDouble(),
                           rand.NextDouble(),
                           rand.NextDouble(),
                           rand.NextDouble());

            // Give it random gradient colors.
            var brush = new LinearGradientBrush(
                            Color.FromScRgb(1f, (float)rand.NextDouble(),
                                                (float)rand.NextDouble(), 
                                                (float)rand.NextDouble()),
                            Color.FromScRgb(1f, (float)rand.NextDouble(),
                                                (float)rand.NextDouble(),
                                                (float)rand.NextDouble()),
                            180 * rand.NextDouble());

            // Add it to the pool of data items.
            DataItems.Add(new { top, left, width, height, data, brush, i });
        }
    }
}

Now lets define the <Style> for our ListBoxItems and the <DataTemplate>s for the three different data “types”.  The first part of the <Style> will be hooking up the Canvas.Top, Canvas.Left, Width, and Height properties for each item, as well as setting the content alignment to Stretch (since by default ListBoxItems left-justify their content).  I’ll also give each ListBoxItem some Padding so that the blue selection highlight is clearly visible around an item when it’s selected.  The second part of the <Style> will be defining our <DataTemplate>s based on the value of the data property.  Since in this case our data already represents simple shapes, our templates are going to be pretty straightforward:

<ListBox.ItemContainerStyle>
    <Style TargetType="ListBoxItem">

        <Setter Property="Canvas.Top" Value="{Binding top}"/>
        <Setter Property="Canvas.Left" Value="{Binding left}"/>
        <Setter Property="Width" Value="{Binding width}"/>
        <Setter Property="Height" Value="{Binding height}"/>
        
        <Setter Property="VerticalContentAlignment" Value="Stretch"/>
        <Setter Property="HorizontalContentAlignment" Value="Stretch"/>
        <Setter Property="Padding" Value="5"/>
        
        <Setter Property="ContentTemplate">
            <Setter.Value>
                <DataTemplate>
                    <Grid>
                        <Path Data="{Binding data}"
                              Stroke="{Binding brush}"
                              StrokeThickness="2"
                              Stretch="Fill"/>
                        <TextBlock Text="{Binding i}"
                                   HorizontalAlignment="Center"
                                   VerticalAlignment="Center"/>
                    </Grid>
                </DataTemplate>
            </Setter.Value>
        </Setter>
        
        <Style.Triggers>
            <DataTrigger Binding="{Binding data}" Value="rectangle">
                <Setter Property="ContentTemplate">
                    <Setter.Value>
                        <DataTemplate>
                            <Grid>
                                <Rectangle Fill="{Binding brush}"
                                           RadiusX="10"
                                           RadiusY="10"/>
                                <TextBlock Text="{Binding i}"
                                           HorizontalAlignment="Center"
                                           VerticalAlignment="Center"/>
                            </Grid>
                        </DataTemplate>
                    </Setter.Value>
                </Setter>
            </DataTrigger>

            <DataTrigger Binding="{Binding data}" Value="ellipse">
                <Setter Property="ContentTemplate">
                    <Setter.Value>
                        <DataTemplate>
                            <Grid>
                                <Ellipse Fill="{Binding brush}"/>
                                <TextBlock Text="{Binding i}"
                                           HorizontalAlignment="Center"
                                           VerticalAlignment="Center"/>
                            </Grid>
                        </DataTemplate>
                    </Setter.Value>
                </Setter>
            </DataTrigger>
            
        </Style.Triggers>
    </Style>
</ListBox.ItemContainerStyle>

And if you run the app with just the code I’ve shown you so far, you already get a pretty cool little canvas application showing 100 random shapes with selection, scroll bars, and keyboard navigation:

(I’ve embedded the sample as an XBAP in an <iframe> above this line, so you should see a live sample and be able to interact with it if you’re using Internet Explorer and have .NET 4.0 installed.)

You may have noticed that if you select an item with the mouse and keep holding down the mouse button while moving the mouse off the canvas, it will start to pan automatically, even though we haven’t hooked up any zooming or panning code yet!  It’s actually the <ListBox> that’s doing this, since it knows how to talk to the IScrollInfo interface that the ZoomableCanvas implements internally.  When you are selecting items by holding the mouse and dragging, the <ListBox> automatically tells the ZoomableCanvas to scroll to the next item.  ZoomableCanvas then does this by setting its Offset property accordingly, which results in the automatic “pan”.

Unfortunately the built-in <ListBox> “panning” is a little clunky, since it does it on an item-by-item basis instead of a pixel-by-pixel basis.  You can of course use the scroll bars instead, but that’s not as cool as dragging the canvas to pan, and there’s no way to zoom.  So lets hook up our own mouse handlers just like we did in ZoomableApplication1, but this time we’ll get even fancier by zooming in and out “around the mouse” instead of the top-left.

If you remember from ZoomableApplication1, we were able to set the Scale and Offset of the ZoomableCanvas by setting its x:Name="MyCanvas" in the XAML.  Unfortunately this feature doesn’t work in WPF when dealing with <ItemsPanelTemplate>s, and the ItemsHost property on ListBox is internal.  So we’ll either have to search down the visual tree to find the canvas, or simply assign it to a local variable when it’s created.  I think the latter is cleaner, so lets add a ZoomableCanvas_Loaded handler and capture it at that point.  And while we’re capturing things, lets capture the mouse this time too:

<ListBox.ItemsPanel>
    <ItemsPanelTemplate>
        <ZoomableCanvas Loaded="ZoomableCanvas_Loaded"/>
    </ItemsPanelTemplate>
</ListBox.ItemsPanel>

ZoomableCanvas MyCanvas;
Point LastMousePosition;

private void ZoomableCanvas_Loaded(object sender, RoutedEventArgs e)
{
    // Store the canvas in a local variable since x:Name doesn't work.
    MyCanvas = (ZoomableCanvas)sender;
}

protected override void OnPreviewMouseMove(MouseEventArgs e)
{
    var position = e.GetPosition(MyListBox);
    if (e.LeftButton == MouseButtonState.Pressed
        && !(e.OriginalSource is Thumb)) // Don't block the scrollbars.
    {
        CaptureMouse();
        MyCanvas.Offset -= position - LastMousePosition;
        e.Handled = true;
    }
    else
    {
        ReleaseMouseCapture();
    }
    LastMousePosition = position;
}

protected override void OnPreviewMouseWheel(MouseWheelEventArgs e)
{
    var x = Math.Pow(2, e.Delta / 3.0 / Mouse.MouseWheelDeltaForOneLine);
    MyCanvas.Scale *= x;

    // Adjust the offset to make the point under the mouse stay still.
    var position = (Vector)e.GetPosition(MyListBox);
    MyCanvas.Offset = (Point)((Vector)
        (MyCanvas.Offset + position) * x - position);

    e.Handled = true;
}

Before I embed another sample, let’s do one more thing.  Instead of having a fixed set of 100 shapes, let’s dock a <Slider> to the top that lets us increase and decrease the number of shapes at will.  We’ll remove the call to PopulateDataItems(100) that we initially had above, and we’ll call it from the MySlider_ValueChanged instead:

<DockPanel>
    <Slider x:Name="MySlider" DockPanel.Dock="Top"
            Maximum="10000" AutoToolTipPlacement="BottomRight"
            ValueChanged="MySlider_ValueChanged"/>

    <ListBox x:Name="MyListBox">

        . . .

    </ListBox>
</DockPanel>

private void MySlider_ValueChanged(object sender, EventArgs e)
{
    PopulateDataItems((int)MySlider.Value);
}

The thing to notice here (other than the fun panning and zooming) is that no matter how many shapes you create, you can scroll back and forth extremely smoothly as long as your are zoomed in enough so that only a few hundred shapes are on your screen at a time.  This is because of the built-in UI virtualization in ZoomableCanvas.  

It’s nice that we get virtualization for free without doing any extra work, but you probably noticed it took a really long time to get that slider all the way over to the right.  This is because the ZoomableCanvas had to create and measure each of those shapes (albeit just once) in order to determine where they would end up on the canvas.  We can skip this step by having our ItemsSource implement ZoomableCanvas.ISpatialItemsSource.  This will cause the ZoomableCanvas to ask our ItemsSource which items are in view instead of the ZoomableCanvas keeping track of them internally, which means the ZoomableCanvas doesn’t need to know where all the shapes are up front.  It’s pretty easy to implement ISpatialItemsSource; you just need to know how to figure out the index of the items that intersect a given Rect, and you need to know the overall bounding box (extent) of all the items in your set.  Since all of our items are in a grid-like pattern, it’s very quick to determine both of these things.  I’m just going to create a new class that derives from ObservableCollection<object> (since that’s what we were using before), and implement ISpatialItemsSource by using some grid-like math:

public class GridLikeItemsSource : 
    ObservableCollection<object>,
    ZoomableCanvas.ISpatialItemsSource
{
    public Rect Extent
    {
        get
        {
            var sqrt = Math.Sqrt(Count);
            return new Rect(0, 0, 
                100 * (Math.Ceiling(sqrt)), 
                100 * (Math.Round(sqrt)));
        }
    }

    public IEnumerable<int> Query(Rect rectangle)
    {
        rectangle.Intersect(Extent);

        var top = Math.Floor(rectangle.Top / 100);
        var left = Math.Floor(rectangle.Left / 100);
        var right = Math.Ceiling(rectangle.Right / 100);
        var bottom = Math.Ceiling(rectangle.Bottom / 100);

        for (var x = Math.Max(left, 0); x <= right; x++)
        {
            for (var y = Math.Max(top, 0); y <= bottom; y++)
            {
                var i = x > y ?
                    Math.Pow(x, 2) + y :
                    Math.Pow(y, 2) + 2 * y - x;
                if (i < Count)
                {
                    yield return (int)i;
                }
            }
        }
    }

    public event EventHandler ExtentChanged;

    public event EventHandler QueryInvalidated;
}

I’ll also change the corresponding type in MainWindow, but I’ll leave everything else the same.

public MainWindow()
{
    InitializeComponent();

    DataItems = new GridLikeItemsSource();
    MyListBox.ItemsSource = DataItems;
}

private GridLikeItemsSource DataItems;

Here is the sample again, but you should be able to feel a big difference when sliding the slider across:

Moving the slider all the way to the right is still not instantaneous, but that’s because it still takes a little bit of time to create each data item and add it to the ObservableCollection.  It’s also not free since each item takes a little bit of RAM as well.  This is fine for 10,000 items, but what about 100,000?  Or a million?  Or a billion?  What if I wanted to show a node-link diagram for every web page on the internet?  There’s over 150 billion web pages according to www.archive.org, and who knows how many links!  You wouldn’t even be able to fit all of that data into RAM!

With data virtualization, you can have a canvas with as many items as you want.  Well, ok, that’s not entirely true.  You can actually only have up to 2,147,483,647 items per canvas, because that’s the upper limit of the int indexer on IList.  The key to data virtualization is that when you implement ISpatialItemsSource, your list is never actually enumerated.  The only thing that is called is ISpatialItemsSource.Query (and Extent), so your list is only asked for the items that are actually on the screen.  So instead of deriving from ObservableCollection and adding items to it when the slider value changes, let’s implement a custom IList that fetches items on-demand.

public class GridLikeItemsSource : 
    IList,
    ZoomableCanvas.ISpatialItemsSource
{
    public Rect Extent
    {
        . . . (same code as above) . . .
    }

    public IEnumerable<int> Query(Rect rectangle)
    {
        . . . (same code as above) . . .
    }

    public event EventHandler ExtentChanged;

    public event EventHandler QueryInvalidated;

    private int count;
    
    public int Count
    {
        get
        {
            return count;
        }
        set
        {
            count = value;
            ExtentChanged(this, EventArgs.Empty);
            QueryInvalidated(this, EventArgs.Empty);
        }
    }

    public object this[int i]
    {
        get
        {
            // Give the item a deterministic seed.
            var rand = new Random(i);
            var sqrt = (int)Math.Sqrt(i);

            // With a random width and height between 50 and 100.

            . . . (same code as PopulateDataItems above) . . .

            return new { top, left, width, height, data, brush, i };
        }
        set
        {
        }
    }

    #region Irrelevant IList Members . . .
}

Notice that this time I’m actually raising the ExtentChanged and QueryInvalidated events.  This wasn’t necessary before because ObservableCollection already raises enough events to broadcast to the world that everything has changed.  But now that we’re trimmed down, we raise ExtentChanged to let the ZoomableCanvas know to update its scrollbar information, and we raise QueryInvalidated to let the ZoomableCanvas know to call Query again.  If we wanted to be totally optimized then we would only call QueryInvalidated when we know that another call to Query with the same Rect would give different results.  But our Query method runs practically instantaneously so it doesn’t really matter if it’s called tons of times.

Now lets update our <Slider> to have a million items and replace the PopulateDataItems code with a simple setter to DataItems.Count:

<Slider x:Name="MySlider" DockPanel.Dock="Top"
        Maximum="1000000" AutoToolTipPlacement="BottomRight"
        ValueChanged="MySlider_ValueChanged"/>

private void MySlider_ValueChanged(object sender, EventArgs e)
{
    DataItems.Count = (int)MySlider.Value;
}

One more thing before we run the app.  I know that you’re going to slide the slider all the way to the right (which will be instantaneous with our data-virtualized IList), but then the first thing you’re going to do after that is zoom out!  But we haven’t done anything to prevent WPF from trying to show all 1,000,000 shapes, so the app will hang and/or crash just like every other VirtualCanvas sample out there, and my blog will probably crash and burn right along with it.  So before I let you unleash your mouse wheel on the next sample, I’m going to put in some safeguards.

WPF can only handle between about 1,000 to 10,000 primitives on the screen while still remaining responsive, depending on your CPU and graphics card.  10,000 primitives isn’t a lot when you realize that even our simple shapes use 5 primitives each (you can use Snoop to see why).  ZoomableCanvas doesn’t have any magic to get around that, but it does have convenient ways to limit the number of elements on the screen, and ways to control the time it takes to create them.  The first property, RealizationLimit, is the maximum number of elements that ZoomableCanvas will try to maintain on the screen at any given time.  The second property, RealizationRate, is the number of elements that ZoomableCanvas will create and/or dispose in a single “batch”.  This is really only useful when you combine it with the third property, RealizationPriority, which controls when these batches take place.  By default the RealizationPriority is DispatcherPriority.Normal, which basically means it performs the batches right away even if it means hanging the UI.

For our particular sample, I’m going to set RealizationLimit to 1000, RealizationRate to 10, and RealizationPriority to Background.  This means that the ZoomableCanvas will try to have a maximum of 1000 elements on the screen at a time (5,000 primitives in our case), it will create and dispose 10 of them at a time until it gets to that number, and it will process input events like the mouse and keyboard before each batch.  This will let the app remain responsive and allow you to pan and zoom way out, with the tradeoff being that you’ll see a bit of a “flicker” while the shapes are being created and disposed and you’ll see big gaps between the 1,000 items on the screen.

Here is the code:

<ListBox.ItemsPanel>
    <ItemsPanelTemplate>
        <ZoomableCanvas Loaded="ZoomableCanvas_Loaded"
                        RealizationLimit="1000"
                        RealizationRate="10"
                        RealizationPriority="Background"/>
    </ItemsPanelTemplate>
</ListBox.ItemsPanel>

And finally, here is the app!

The distribution of elements on the canvas when there is a RealizationLimit is not random.  When you’re not implementing ISpatialItemsSource yourself, the ZoomableCanvas will by default show larger items (width + height) before showing smaller ones.  In other words, if your RealizationLimit is 1000, the ZoomableCanvas will create elements for the 1000 biggest items (within the viewbox) first, and then it will dispose the rest.  But if you are implementing ISpatialItemsSource yourself, then ZoomableCanvas will create elements for the first 1000 items that you return from the Query method, in the order you return them, so it’s up to you to prioritize which ones show up on the canvas first.  You’re welcome to use PriorityQuadTree if you’d like, but for this particular sample I was able to come up with a more efficient technique to prioritize evenly-distributed items since the elements are in a grid-like pattern:

public IEnumerable<int> Query(Rect rectangle)
{
    rectangle.Intersect(Extent);

    var top = Math.Floor(rectangle.Top / 100);
    var left = Math.Floor(rectangle.Left / 100);
    var right = Math.Ceiling(rectangle.Right / 100);
    var bottom = Math.Ceiling(rectangle.Bottom / 100);
    var width = Math.Max(right - left, 0);
    var height = Math.Max(bottom - top, 0);

    foreach (var cell in Quadivide(new Rect(left, top, width, height)))
    {
        var x = cell.X;
        var y = cell.Y;
        var i = x > y ?
            Math.Pow(x, 2) + y :
            Math.Pow(y, 2) + 2 * y - x;
        if (i < Count)
        {
            yield return (int)i;
        }
    }
}

private IEnumerable<Point> Quadivide(Rect area)
{
    if (area.Width > 0 && area.Height > 0)
    {
        var center = area.GetCenter();
        var x = Math.Floor(center.X);
        var y = Math.Floor(center.Y);
        yield return new Point(x, y);

        var quad1 = new Rect(area.TopLeft, new Point(x, y + 1));
        var quad2 = new Rect(area.TopRight, new Point(x, y));
        var quad3 = new Rect(area.BottomLeft, new Point(x + 1, y + 1));
        var quad4 = new Rect(area.BottomRight, new Point(x + 1, y));

        var quads = new Queue<IEnumerator<Point>>();
        quads.Enqueue(Quadivide(quad1).GetEnumerator());
        quads.Enqueue(Quadivide(quad2).GetEnumerator());
        quads.Enqueue(Quadivide(quad3).GetEnumerator());
        quads.Enqueue(Quadivide(quad4).GetEnumerator());
        while (quads.Count > 0)
        {
            var quad = quads.Dequeue();
            if (quad.MoveNext())
            {
                yield return quad.Current;
                quads.Enqueue(quad);
            }
        }
    }
}

So there you have it!  Hopefully this (rather long) blog post gave you a good idea of how use the built-in UI virtualization, how to implement your own data data virtualization, and how to put on limits to keep your app responsive as a last resort.  I’ve attached the final ZoomableApplication2 project to this post.  Phew!

ZoomableApplication2.zip

The next thing I’ll do is get rid of the default <Grid> and replace it with a <ZoomableCanvas> with some children, like this:

<Window x:Class="ZoomableApplication1.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525">
    <ZoomableCanvas>
        <Rectangle Canvas.Top="0" Canvas.Left="0"
                   Width="30" Height="40" Fill="LightBlue"/>

        <Button    Canvas.Top="10" Canvas.Left="40"
                   Content="Hello, ZoomableCanvas!"/>

        <Calendar  Canvas.Top="40" Canvas.Left="10"
                   DisplayDate="6/18/2008"/>
    </ZoomableCanvas>
</Window>

And as you’d expect with a regular <Canvas>, this is the result:

(I’ve embedded the sample as an XBAP in an <iframe> above this line, so you should see a live sample and be able to interact with it if you’re using Internet Explorer and have .NET 4.0 installed.)

So far this is no different from a regular <Canvas>, so lets start off by setting the Scale and Offset directly in the XAML, like this:

    <ZoomableCanvas Scale=".5" Offset="-100,0">

You’ll notice that the elements have shrunk by half and moved to the right by 100 pixels.  But this is not normally how you’d set Scale and Offset.  Normally you’d change them dynamically; for example you might want to set Scale with a zoom slider, like this:

    <DockPanel>
        <Slider Minimum=".1" Maximum="2" Orientation="Vertical"
                Value="{Binding ElementName=MyCanvas, Path=Scale}"/>

        <ZoomableCanvas x:Name="MyCanvas">
            <Rectangle Canvas.Top="0" Canvas.Left="0"
                Width="30" Height="40" Fill="LightBlue"/>

            <Button    Canvas.Top="10" Canvas.Left="40"
                Content="Hello, ZoomableCanvas!"/>

            <Calendar  Canvas.Top="40" Canvas.Left="10"
                DisplayDate="6/18/2008"/>
        </ZoomableCanvas>
    </DockPanel>

But of course it’s much more natural to use the mouse wheel to zoom and click and drag to pan.  So let’s override the OnMouseWheel and OnMouseMove events on our Window to control the Scale and Offset as well:

protected override void OnMouseWheel(MouseWheelEventArgs e)
{
    var x = Math.Pow(2, e.Delta / 3.0 / Mouse.MouseWheelDeltaForOneLine);
    MyCanvas.Scale *= x;
}

private Point LastMousePosition;

protected override void OnMouseMove(MouseEventArgs e)
{
    var position = e.GetPosition(this);
    if (e.LeftButton == MouseButtonState.Pressed)
    {
        MyCanvas.Offset -= position - LastMousePosition;
    }
    LastMousePosition = position;
}

The OnMouseWheel method takes the distance that the mouse wheel was turned (which is given by e.Delta), divides it by 3 since by default Windows expects to scroll 3 “lines” per tick, and then divides that amount by the amount of Delta that represents each “line”.  That value could be negative since e.Delta is negative when the mouse wheel is rotated backwards, so raising 2 to that power gives us a nice value above or below 1.0, but never below 0.  Then we simply adjust the scale by multiplying it by that amount.

The OnMouseMove method is about equally as simple.  It just keeps track of the last mouse position, and if the mouse button is held down when you drag the mouse then it adjusts Offset by the amount you dragged it.  Of course, we aren’t setting e.Handled = true or doing any mouse capturing in this tiny example, so you get some interesting interactions with other elements on the canvas that try to handle the mouse as well.  Try dragging the zoom slider remaining from the previous example for an interesting effect!

That’s about it for Scale and Offset.  In the upcoming posts I’ll show some more examples of mouse handling, including having the canvas zoom in and out “around the mouse”, which means that the point under the mouse cursor will stay “fixed” while the other points on the canvas move in and out relative to it as you zoom.  You can take a look at Bing maps to see what I mean.

In the mean time, I’ve attached the ZoomableApplication1 project to this post, and it includes the class library that I mentioned in the beginning.  The class library simply contains the source code files listed at bottom of this post: http://blogs.msdn.com/b/kaelr/archive/2010/07/29/zoomablecanvas.aspx, but if you don’t want to go download all the source files yourself then you can just grab the ZoomableCanvas.dll assembly from here since I’ve already done the work for you.  Enjoy!

ZoomableApplication1.zip

The reason that Scale only sometimes controls the size of the elements on screen is because the ZoomableCanvas has a bool property called ApplyTransform, and if ApplyTransform = false then the ZoomableCanvas will not affect the size of elements at all.  ApplyTransform is set to true by default, which means you get scaling automatically for free, but sometimes you may wish to turn it off so you can do custom effects instead, such as semantic zoom.  When ApplyTransform = false, the Scale will still affect the position of the elements (i.e. they get closer together as you zoom out, and farther apart as you zoom in), but changing the size of the elements is up to you.  If you forget to change the size, then you’ll end up with your elements appearing to “float above” the canvas, and in fact the search results and definition labels in Code Canvas do exactly that on purpose.  Check out the various Code Canvas videos to see that effect in action.

ZoomableCanvas is similar to Chris Lovett’s VirtualCanvas, but it has some important differences.  The main difference is that ZoomableCanvas is meant to feel like a built-in WPF component, just like the default Canvas, and so it supports the three most common methods of populating a Panel in WPF:

1. Adding children directly to the canvas, either via the Children property or via XAML.
2. Adding children with the Items property of a parent ItemsControl (like a ListBox).
3. Binding to a data source with the ItemsSource property of a parent ItemsControl.

In the first case, you can use it exactly how you’d use a normal Canvas:

    <ZoomableCanvas>
        <Rectangle Canvas.Top="10" Canvas.Left="20"
                   Width="30" Height="40" Fill="LightBlue"/>

        <Button    Canvas.Top="10" Canvas.Left="60"
                   Content="Hello, ZoomableCanvas!"/>

        <Calendar  Canvas.Top="40" Canvas.Left="60"
                   DisplayDate="6/18/2008"/>
    </ZoomableCanvas>

In the second case, you can also use it in exactly the same way that you would if you were to use a normal Canvas within an ItemsControl:

    <ListBox>
        <ListBox.ItemsPanel>
            <ItemsPanelTemplate>
                <ZoomableCanvas/>
            </ItemsPanelTemplate>
        </ListBox.ItemsPanel>

        <ListBox.ItemContainerStyle>
            <Style>
                <Setter Property="Canvas.Top"
                        Value="{Binding Path=Content.(Canvas.Top),
                                RelativeSource={RelativeSource Self}}"/>
                
                <Setter Property="Canvas.Left"
                        Value="{Binding Path=Content.(Canvas.Left),
                                RelativeSource={RelativeSource Self}}"/>
            </Style>
        </ListBox.ItemContainerStyle>

        <ListBox.Items>
            <Rectangle Canvas.Top="10" Canvas.Left="20"
                       Width="30" Height="40" Fill="LightBlue"/>

            <Button    Canvas.Top="10" Canvas.Left="60"
                       Content="Hello, ZoomableCanvas!"/>

            <Calendar  Canvas.Top="40" Canvas.Left="60"
                       DisplayDate="6/18/2008"/>
        </ListBox.Items>
    </ListBox>

The fact that you have to set <ListBox.ItemContainerStyle> is a bit ugly, but it’s not something caused by ZoomableCanvas.  Since ItemsControls always wrap their items in ContentPresenters, you end up with an extra level of indirection between the item and the Panel, so you have to use a <Style> on the ContentPresenter to make the connection.  This is a well-known issue for anyone who has ever tried to use a Grid or a DockPanel in a ListBox, for example.

The third case is probably the most likely, and it’s basically the same as the last one except that you set ItemsSource="{Binding ...}" instead of specifying the <ListBox.Items> directly.  This is also the only way to get data virtualization.

Speaking of virtualization, ZoomableCanvas supports both kinds of virtualization: UI virtualization and data virtualization.  UI virtualization is when you have lots of data items loaded into memory (plain .NET objects, not Visuals), but you only create the Visuals for those items that can be seen on the screen.  Data virtualization is when you don’t even load all of the data items into memory; you only load a subset of the data, but you “pretend” that the full data set is there.  You can even have both kinds of virtualization going on at once.  For example, imagine you’re creating an application that draws a social networking graph of all of the users on the Internet.  Your back-end database would likely have millions (or billions) of nodes and edges, but someone using a 24” monitor would realistically only be able to see a few thousand at once.  Your users probably don’t have enough RAM to load the entire dataset into memory, but they probably don’t have enough bandwidth to issue thousands of queries on-the-fly as they pan and zoom around.  So you can use data virtualization to fetch and cache only some moderate portion of the full data set (e.g. a couple hundred thousand nodes around the neighborhood they’re viewing), and use UI virtualization to only create Visuals for the few hundreds of nodes that are actually on their screen.

ZoomableCanvas provides some default UI virtualization for free out of the box by using VirtualPanel and PriorityQuadTree, both posted previously on my blog.  You can also improve the performance even further by having your data source implement ZoomableCanvas.ISpatialItemsSource.  When the value you gave to ItemsSource implements ISpatialItemsSource, the ZoomableCanvas will call the Query method instead of enumerating over the whole list.  The Query method simply takes the current viewbox as a parameter and returns the indexes of the items that fall within that viewbox.  This is also known as a spatial index, and it’s what you can use to implement data virtualization as well.  If you don’t implement ISpatialItemsSource, then the ZoomableCanvas will build its own spatial index internally using the PriorityQuadTree, but it will have to enumerate the whole data list and create each Visual one time to do it.  The Visuals that are off-screen will quickly be re-virtualized, but the ZoomableCanvas needs to create them temporarily so that it can inspect the Canvas.Top and Canvas.Left properties to see where the item needs to go.  By having your data source implement ISpatialItemsSource, you can completely bypass this step.

A final thing to keep in mind is that, although ZoomableCanvas provides easy-to-use properties like Offset, Scale, and even Viewbox, Stretch, and StretchDirection, it does not provide any default keyboard or mouse gesture handling.  The ZoomableCanvas is meant to be like any other built-in WPF component, and it’s expected that each application will be responsible for handling their own user input differently.  For example, the built-in Canvas doesn’t provide mouse handling to let users drag items around on the canvas, even though that’s what most people do with it.  In the same vein, ZoomableCanvas doesn’t provide any mouse handling for dragging items or panning or zooming.  But I will be posting a blog series on the various cool things you can do with ZoomableCanvas, and you will quickly see that adding these things is a piece of cake.

I’ve attached the full source code for ZoomableCanvas to this post, but it relies on the VirtualPanel, PriorityQuadTree, MathExtensions, and LinkedListExtensions from the other posts on my blog.  Once you’ve downloaded those files and put them all in a class library, you can throw in some XmlnsDefinitions and you’ll have a ZoomableCanvas that looks like it came straight from WPF!  Enjoy!

ZoomableCanvas.cs