C# UWP Reading pixels from the screen

Question

Dear,
I am writing a program in C# for UWP. I have a canvas component and I add colored rectangles, lines and ellipses on the canvas. I also add a Textbox component with a few letters. Any new item is added on top of the previous. When I have composed it I want to access every single pixel and modify it. I use this code to get all pixels into a byte array:

//This function output a byte array that uses 4 bytes per pixel
RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(this.gridCanvas);
var renderBitmapPixels = await renderTargetBitmap.GetPixelsAsync(); // return an IBuffer stream

        DataReader dataReader = DataReader.FromBuffer(renderBitmapPixels);  
        byte[] pixels = new byte[renderBitmapPixels.Length];  
        dataReader.ReadBytes(pixels);  

but, the text added by the Textbox is not in the pixels, while all rectangles, lines and ellipses are.

How do I get the text included in the pixels?
How do I get the pixels as I see them on the screen?

Regards,
Jorgen Sandberg

Answer 1

Dear Roy,

Thank you for showing interest in this issue.

My application is to find the dominating color per cell in a grid drawn on the canvas. My code for this is:

int xPixel1 = 0;
int yPixel1 = 0;
int pixIndex = 0;
int[] colourHits = new int[Data.numColours_int];
int[] colourDist = new int[Data.numColours_int];
int maxHits = 0;
int minDist = 0;
int maxHitsIndex = 0;
int minDistIndex = 0;
Color refColour;
int pixBlue = 0;
int pixRed = 0;
int pixGreen = 0;
int pixAlpha = 0;

        int nLine = 0;  

        // Set dominating colour in each cell  
        for( int j = 0; j < Data.verPix_int; j++)  
        {  
            yPixel1 = y0 + j* Data.gridCellSize_int + 1;  

            for( int i = 0; i < Data.horPix_int; i++)  
            {  
                // Find dominating colour in cell i,j  
                xPixel1 = x0 + i*Data.gridCellSize_int + 1;  

                // Prepare array counting hits for each colour  
                for( int k = 0; k < Data.numColours_int; k++) colourHits[k] = 0;  

                for (int yDeltaPix = 0; yDeltaPix < Data.gridCellSize_int -1; yDeltaPix++)  
                {  
                    for (int xDeltaPix = 0; xDeltaPix < Data.gridCellSize_int -1; xDeltaPix++)  
                    {  
                        pixIndex = 4*( (yPixel1 +  yDeltaPix) * renderTargetBitmap.PixelWidth + (xPixel1 + xDeltaPix) );  
                        pixBlue  = (int)pixels[pixIndex];  
                        pixGreen = (int)pixels[pixIndex + 1];  
                        pixRed   = (int)pixels[pixIndex + 2];  
                        pixAlpha = (int)pixels[pixIndex + 3];  
                                

                        if (i == 2 && j == 2 && yDeltaPix > 20 && xDeltaPix > 20 && nLine < 30)  
                        {  
                            colours4.Text += " pixIndex = " + pixIndex.ToString() +  " xi= " + i.ToString() + " yj= " + j.ToString() + " B= "  
                                + pixBlue.ToString()+ " G= " + pixGreen.ToString()+ " R= " + pixRed.ToString()+ " A= " + pixAlpha.ToString() + "\n";  
                            nLine++;  
                        }  

                        // Find distances squared to ref colours  
                        for ( int iColour = 0; iColour < Data.numColours_int; iColour++)  
                        {  
                            refColour = Data.colours[iColour];  
                            colourDist[iColour] =   ((int)refColour.B - pixBlue)  * ((int)refColour.B - pixBlue)  +  
                                                    ((int)refColour.G - pixGreen) * ((int)refColour.G - pixGreen) +  
                                                    ((int)refColour.R - pixRed)   * ((int)refColour.R - pixRed) +  
                                                    ((int)refColour.A - pixAlpha) * ((int)refColour.A - pixAlpha);  
                        }  

                        // Find closest colour  
                        minDist = colourDist[0];  
                        for( int jColour = 0; jColour < Data.numColours_int; jColour++ )  
                        {  
                            if (colourDist[jColour] <= minDist )  
                            {  
                                minDist = colourDist[jColour];  
                                minDistIndex = jColour;  
                            }  
                        }  

                        // Count colour of minimum distance  
                        colourHits[minDistIndex]++;  
                    }  
                }  
                  
                // Set the closest colour to the colour of the cell  
                maxHits = 0;  
                for( int jColour = 0; jColour < Data.numColours_int; jColour++ )  
                {  
                      
                    if (colourHits[jColour] >= maxHits)  
                    {  
                        maxHits = colourHits[jColour];  
                        maxHitsIndex = jColour;  
                    }  
                }  
                //if( maxHitsIndex > 0 ) colours4.Text += " xi= " + i.ToString() + " yj= " + j.ToString() + " colourHits[" + maxHitsIndex.ToString() +"]= " + colourHits[maxHitsIndex].ToString() + "\n";  
                Data.gridColoursCur_int[i, j] = maxHitsIndex;  
            }  
        }  

For development check purposes I have used:

/*
// For development purposes
int numPixels_int = renderTargetBitmap.PixelWidth * renderTargetBitmap.PixelHeight;
colours4.Text = "Render width = " + renderTargetBitmap.PixelWidth.ToString() + " height = " + renderTargetBitmap.PixelHeight.ToString() +" Number of pixels = " + numPixels_int.ToString() + "\n";
colours4.Text += "gridCanvas width = " + this.gridCanvas.ActualWidth.ToString() + " height = " + this.gridCanvas.ActualHeight.ToString() + "\n";

        colours4.Text += "x0    = " + x0.ToString()    + " y0    = " + y0.ToString() + " cellSize = " + Data.gridCellSize_int.ToString() + "\n";  
        colours4.Text += "xL    = " + xL.ToString()    + " yL    = " + yL.ToString() + "\n";  
        colours4.Text += "pixels.Length = " + pixels.Length.ToString() + "\n";  

        colours4.Text += "\n";  

        for (int dy = -2; dy < 3; dy++)  
        {  
            colours4.Text += "\n";  
            for (int dx = -2; dx < 3; dx++)  
            {  
                int x = x0+Data.gridCellSize_int+dx;  
                int y = y0+Data.gridCellSize_int+dy;  
                int pixIndex = ((y)*renderTargetBitmap.PixelWidth + x)*4;  
                colours4.Text +="x=" + x.ToString() + " y=" + y.ToString() + " B = " + pixels[pixIndex].ToString() +" G = " + pixels[pixIndex+1].ToString()  
                    +" R = " + pixels[pixIndex+2].ToString() +" A = " + pixels[pixIndex+3].ToString() + "\n";  
            }  
        }  
        colours4.Text += "\n";  

        Color lineBlack = Colors.Black;  
        byte g = lineBlack.G;  
        byte b = lineBlack.B;  
        byte r = lineBlack.R;  
        byte a = lineBlack.A;  
        colours4.Text += "Black B = " + b.ToString() +" G = " + g.ToString() +" R = " + r.ToString() +" A = " + a.ToString() + "\n";  

        lineBlack = Colors.White;  
        g = lineBlack.G;  
        b = lineBlack.B;  
        r = lineBlack.R;  
        a = lineBlack.A;  
        colours4.Text += "White B = " + b.ToString() +" G = " + g.ToString() +" R = " + r.ToString() +" A = " + a.ToString() + "\n";  

          
        lineBlack = Colors.Blue;  
        g = lineBlack.G;  
        b = lineBlack.B;  
        r = lineBlack.R;  
        a = lineBlack.A;  
        colours4.Text += "Blue  B = " + b.ToString() +" G = " + g.ToString() +" R = " + r.ToString() +" A = " + a.ToString() + "\n";  

        lineBlack = Colors.Green;  
        g = lineBlack.G;  
        b = lineBlack.B;  
        r = lineBlack.R;  
        a = lineBlack.A;  
        colours4.Text += "Green B = " + b.ToString() +" G = " + g.ToString() +" R = " + r.ToString() +" A = " + a.ToString() + "\n";  

        lineBlack = Colors.Red;  
        g = lineBlack.G;  
        b = lineBlack.B;  
        r = lineBlack.R;  
        a = lineBlack.A;  
        colours4.Text += "Red   B = " + b.ToString() +" G = " + g.ToString() +" R = " + r.ToString() +" A = " + a.ToString() + "\n";  
          
        colours.Text += "\n";  

        for (int iColour = 0; iColour < Data.numColours_int; iColour++)  
        {  
            lineBlack = Data.colours[iColour];  
            g = lineBlack.G;  
            b = lineBlack.B;  
            r = lineBlack.R;  
            a = lineBlack.A;  
            colours4.Text += "Colour: " + iColour.ToString() + " B = " + b.ToString() +" G = " + g.ToString() +" R = " + r.ToString() +" A = " + a.ToString() + "\n";  
        }  
*/  

I hope you can make some sense of it. I know where the text is on the canvas (I can see it) hence the code for development purposes looks at only a small fraction of the canvas concentrated around the int x and int y variables. The code to find the dominating color never identified any cell dominated by the text, although there are several when I look at my screen.

Regards,
Jorgen

Answer 2

OK, I will try.

Regards,
Jorgen

Answer 3

Dear Roy,

I have reduced my code to the essential. Here it is:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices.WindowsRuntime;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
using Windows.UI.ViewManagement;
using Windows.UI.Xaml.Shapes;

using Windows.Graphics.Imaging; // To get size in pixels
using Windows.Graphics;
using System.Numerics;

using Windows.UI; // Added to include Color
using Windows.UI.Text; // Added to use text definitions
using Windows.UI.Xaml.Media.Imaging;
using Windows.Storage.Streams;
using Windows.Security.Cryptography.Core;
using Windows.UI.Popups;

// The Blank Page item template is documented at https://go.microsoft.com/fwlink/?LinkId=402352&clcid=0x409

namespace IconCreator
{
/// <summary>
/// An empty page that can be used on its own or navigated to within a Frame.
/// </summary>
public sealed partial class MainPage : Page
{
public static int countLoaded = 0;
public static int countSizeChanged = 0;
public static int countNavigatedTo = 0;
public static int countSaveButton = 0;

    private static int sideColourSquares;  
    private static int x0, y0, xL, yL;  
      


    public MainPage()  
    {  
        this.InitializeComponent();  
        ApplicationView.PreferredLaunchViewSize = new Size(2000, 1200);  
        ApplicationView.PreferredLaunchWindowingMode = ApplicationViewWindowingMode.PreferredLaunchViewSize;  

          
    }  

    private void LayoutRoot_SizeChanged(object sender, SizeChangedEventArgs e)  
    {  
        Data.canvasHeight_int = (int)gridCanvas.ActualHeight;  
        Data.canvasWidth_int  = (int)gridCanvas.ActualWidth;  

        Setup();  
    }  

    //=====================================================================================================================  

    private void toRaster_Click(object sender, RoutedEventArgs e)  
    {  
        // Transform the current content of the grid to a raster  
        ShowTextRaster();  
    }  
      
    //===================================================================================================================================================================================================  

    private void Setup()  
    {  
        int ix = 8;  
        int iy = 12;  

        comments.Text = "FromSetup(): \n";   
        comments.Text += " Canvas height = " + Data.canvasHeight_int.ToString() + " width = " + Data.canvasWidth_int.ToString() + "\n";  

        Data.horPix_int = 10;  
        Data.verPix_int = 15;  
        Data.CreateColourArray(2);  
        Data.colours[0] = Color.FromArgb( (byte)255, (byte)0, (byte)255, (byte)0 );  
        Data.colours[1] = Color.FromArgb( (byte)255, (byte)255, (byte)0, (byte)0 );  
        Data.CreateGridColourArray();  
        Data.colourActive_int = 1;  

        ShowGrid();  
        Data.gridColoursRef_int[ix, iy] = 1;  
        PaintRefGridCells();  

        Data.textGridCellHor_int = 1;  
        Data.textGridCellVer_int = 1;  
        Data.textText_str = "S";  
        Data.textFontFamily_str = "Bahnschrift";  
        Data.textFontSize_int = 700;  

        ShowTextRaw();  
    }  


     
    private void DrawColourSquare(int iX, int iY, int iColour)  
    {  
        Rectangle rectangle = new Rectangle();  
        rectangle.Height = sideColourSquares;  
        rectangle.Width = sideColourSquares;  
        //Grid.SetRow(rectangle, 1);  
        Canvas.SetLeft(rectangle, iX);  
        Canvas.SetTop(rectangle, iY);  
        rectangle.HorizontalAlignment= HorizontalAlignment.Left;  
        rectangle.VerticalAlignment= VerticalAlignment.Top;  
        //rectangle.Margin = new Thickness(20, 30 + (sideSpace+sideColour)*i, 0, 0);  
        rectangle.Fill = new SolidColorBrush(Data.colours[iColour]);  
        rectangle.Stroke = new SolidColorBrush(Colors.Black);  
        rectangle.StrokeThickness = 2;  
        //setupGrid.Children.Add(rectangle);  
        gridCanvas.Children.Add(rectangle);  
    }  
      
    private void ShowGrid()  
    {  
        // Draw the grid  
        int horSize = 0;  
        int verSize = 0;  

        // Find the stepsize between lines  
        horSize = (Data.canvasWidth_int - 300)/Data.horPix_int;  
        verSize = (Data.canvasHeight_int - 200)/Data.verPix_int;  
        if (horSize > verSize)  
        {  
            Data.gridCellSize_int = verSize;  
        }  
        else  
        {  
            Data.gridCellSize_int = horSize;  
        }  

        // Find the size of the grid  
        horSize = Data.gridCellSize_int*Data.horPix_int;  
        verSize = Data.gridCellSize_int*Data.verPix_int;  

        // Position the grid  
        // Top left corner  
        x0 = 200 + (Data.canvasWidth_int - 200 - horSize)/2;  
        y0 = (Data.canvasHeight_int - verSize)/2;  

        // Bottom right corner  
        xL = x0 + horSize;  
        yL = y0 + verSize;  

        // Draw the vertical lines  
        for (int i = 0; i <= Data.horPix_int; i++)  
        {  
            DrawLine(x0 + i*Data.gridCellSize_int, y0, x0 + i*Data.gridCellSize_int, yL);  
        }  

        // Draw the horizontal lines  
        for (int i = 0; i <= Data.verPix_int; i++)  
        {  
            DrawLine(x0, y0 + i*Data.gridCellSize_int, xL, y0 + i*Data.gridCellSize_int);  
        }  
          
    }  
             
    private void PaintCurGridCells()  
    {  
        // Paint the Cur grid cells  
        for (int jGrid = 0; jGrid < Data.verPix_int; jGrid++)  
        {  
            for (int iGrid = 0; iGrid < Data.horPix_int; iGrid++)  
            {  
                PaintCell(x0 + iGrid*Data.gridCellSize_int, y0 + jGrid*Data.gridCellSize_int, Data.gridColoursCur_int[iGrid, jGrid]);  
            }  
        }  
    }  

    private void PaintRefGridCells()  
    {  
        // Paint the Ref grid cells  
        for (int jGrid = 0; jGrid < Data.verPix_int; jGrid++)  
        {  
            for (int iGrid = 0; iGrid < Data.horPix_int; iGrid++)  
            {  
                PaintCell(x0 + iGrid*Data.gridCellSize_int, y0 + jGrid*Data.gridCellSize_int, Data.gridColoursRef_int[iGrid, jGrid]);  
            }  
        }  
    }  


    private void ShowTextRaw()  
    {  
        // The text is shown on top of the Ref grid.  
        ShowGrid();  
        PaintRefGridCells();  

        // Copy Data.gridColoursRef_int[iGrid, jGrid] to Data.gridColoursCur_int[iGrid, jGrid]  
        CopyGridFillRefToCur();  

        //  
        TextBlock canvasText = new TextBlock();  
        canvasText.Text = Data.textText_str;  
        canvasText.HorizontalAlignment = HorizontalAlignment.Left;  
        canvasText.VerticalAlignment = VerticalAlignment.Top;  
        double xPos = x0 + Data.textGridCellHor_int*Data.gridCellSize_int;  
        double yPos = y0 + Data.textGridCellVer_int*Data.gridCellSize_int;  
        if( xPos < x0 ) xPos = x0;  
        if( yPos < y0 ) yPos = y0;  
        if( xPos > xL ) xPos = xL;  
        if( yPos > yL ) yPos = yL;  
        canvasText.Margin = new Thickness(xPos, yPos, 0, 0);  
        canvasText.Foreground = new SolidColorBrush(Data.colours[Data.colourActive_int]);  
        canvasText.FontFamily = new FontFamily(Data.textFontFamily_str);  
        canvasText.FontSize = Data.textFontSize_int;  
        //canvasText.FontWeight = FontWeights.Bold;     
        //canvasText.FontStyle  = FontStyle.Normal;  
        gridCanvas.Children.Add(canvasText);  
                                         
    }  

    private void ShowTextRaster()  
    {  
        // Transform the image containing the text to raster  
        // The image on raster is returned in the Data.gridColoursCur_int[,]  
        ImageOnRaster();  

        // Paint the Cur grid cells  
        ShowGrid();  
        PaintCurGridCells();  
    }  

    private void DrawLine(int x1, int y1, int x2, int y2)  
    {  
        // Draw a line on the canvas from the point (x1,y1) to the point (x2,y2)  
        Line myLine = new Line();  
        myLine.X1 = x1;  
        myLine.Y1 = y1;  
        myLine.X2 = x2;  
        myLine.Y2 = y2;  
        myLine.Stroke = new SolidColorBrush(Colors.Black);  
        myLine.StrokeThickness = 1;  
        gridCanvas.Children.Add(myLine);  
        return;  
    }  
            
    private void PaintCell(int iX, int iY, int iColour)  
    {  
        // The input parameters iX and iY are the coordinates of the crossing of  
        // the lines of the grid  
        Rectangle rectangle = new Rectangle();  
        rectangle.Height = Data.gridCellSize_int-1;  
        rectangle.Width  = Data.gridCellSize_int-1;  
        Canvas.SetLeft(rectangle, iX+1);  
        Canvas.SetTop(rectangle, iY+1);  
        rectangle.HorizontalAlignment= HorizontalAlignment.Left;  
        rectangle.VerticalAlignment= VerticalAlignment.Top;  
        rectangle.Fill = new SolidColorBrush(Data.colours[iColour]);  
        gridCanvas.Children.Add(rectangle);  
    }  

    private void CopyGridFillCurToRef()  
    {  
        for (int j = 0; j < Data.verPix_int; j++)  
        {  
            for (int i = 0; i < Data.horPix_int; i++)  
            {  
                Data.gridColoursRef_int[i, j] =  Data.gridColoursCur_int[i, j];  
            }  
        }  
        // Freeze the icon   
        Data.iconPositionFixed_bool = true;  
    }  

    private void CopyGridFillRefToCur()  
    {  
        for (int j = 0; j < Data.verPix_int; j++)  
        {  
            for (int i = 0; i < Data.horPix_int; i++)  
            {  
                Data.gridColoursCur_int[i, j] =  Data.gridColoursRef_int[i, j];  
            }  
        }  
    }  

    private async void ImageOnRaster()  
    {  
        // This function output a byte array that uses 4 bytes per pixel  
        RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();  
        await renderTargetBitmap.RenderAsync(this.gridCanvas);  
        var renderBitmapPixels = await renderTargetBitmap.GetPixelsAsync(); // return an IBuffer stream  

        DataReader dataReader = DataReader.FromBuffer(renderBitmapPixels);  
        byte[] pixels = new byte[renderBitmapPixels.Length];  
        dataReader.ReadBytes(pixels);  

        comments.Text += "From ImageOnRater() \n";  
        comments.Text += " renderTargetBitmap.PixelWidth = " + renderTargetBitmap.PixelWidth.ToString() + "\n";  
        comments.Text += " x0    = " + x0.ToString()    + " y0    = " + y0.ToString() + " cellSize = " + Data.gridCellSize_int.ToString() + "\n";  
        comments.Text += " xL    = " + xL.ToString()    + " yL    = " + yL.ToString() + "\n\n";  
          

        int xPixel1 = 0;  
        int yPixel1 = 0;  
        int pixIndex = 0;  
        int[] colourHits = new int[Data.numColours_int];  
        int[] colourDist = new int[Data.numColours_int];  
        int maxHits = 0;  
        int minDist = 0;  
        int maxHitsIndex = 0;  
        int minDistIndex = 0;  
        Color refColour;  
        int pixBlue = 0;  
        int pixRed = 0;  
        int pixGreen = 0;  
        int pixAlpha = 0;  
                    
        // Set dominating colour in each cell  
        for( int j = 0; j < Data.verPix_int; j++)  
        {  
            yPixel1 = y0 + j* Data.gridCellSize_int + 1;  

            for( int i = 0; i < Data.horPix_int; i++)  
            {  
                // Find dominating colour in cell i,j  
                xPixel1 = x0 + i*Data.gridCellSize_int + 1;  

                // Prepare array counting hits for each colour  
                for( int k = 0; k < Data.numColours_int; k++) colourHits[k] = 0;  

                for (int yDeltaPix = 0; yDeltaPix < Data.gridCellSize_int -1; yDeltaPix++)  
                {  
                    for (int xDeltaPix = 0; xDeltaPix < Data.gridCellSize_int -1; xDeltaPix++)  
                    {  
                        pixIndex = 4*( (yPixel1 +  yDeltaPix) * renderTargetBitmap.PixelWidth + (xPixel1 + xDeltaPix) );  
                        pixBlue  = (int)pixels[pixIndex];  
                        pixGreen = (int)pixels[pixIndex + 1];  
                        pixRed   = (int)pixels[pixIndex + 2];  
                        pixAlpha = (int)pixels[pixIndex + 3];  
                                              
                        // Find distances squared to ref colours  
                        for ( int iColour = 0; iColour < Data.numColours_int; iColour++)  
                        {  
                            refColour = Data.colours[iColour];  
                            colourDist[iColour] =   ((int)refColour.B - pixBlue)  * ((int)refColour.B - pixBlue)  +  
                                                    ((int)refColour.G - pixGreen) * ((int)refColour.G - pixGreen) +  
                                                    ((int)refColour.R - pixRed)   * ((int)refColour.R - pixRed) +  
                                                    ((int)refColour.A - pixAlpha) * ((int)refColour.A - pixAlpha);  
                        }  

                        // Find closest colour  
                        minDist = colourDist[0];  
                        for( int jColour = 0; jColour < Data.numColours_int; jColour++ )  
                        {  
                            if (colourDist[jColour] <= minDist )  
                            {  
                                minDist = colourDist[jColour];  
                                minDistIndex = jColour;  
                            }  
                        }  

                        // Count colour of minimum distance  
                        colourHits[minDistIndex]++;  
                    }  
                }  
                  
                // Set the closest colour to the colour of the cell  
                maxHits = 0;  
                for( int jColour = 0; jColour < Data.numColours_int; jColour++ )  
                {  
                      
                    if (colourHits[jColour] >= maxHits)  
                    {  
                        maxHits = colourHits[jColour];  
                        maxHitsIndex = jColour;  
                    }  
                }  
                  
                if( maxHitsIndex > 0 ) comments.Text += " xi= " + i.ToString() + " yj= " + j.ToString() + " colourHits[" + maxHitsIndex.ToString() +"]= " + colourHits[maxHitsIndex].ToString() + "\n";  
                Data.gridColoursCur_int[i, j] = maxHitsIndex;  
            }  
        }  
    }  
}  

}

<Page
x:Class="IconCreator.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:IconCreator"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d"
Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">

<Grid x:Name="LayoutRoot" SizeChanged="LayoutRoot_SizeChanged">  
    <Grid.ColumnDefinitions>  
        <ColumnDefinition Width="255*"/>  
        <ColumnDefinition/>  
    </Grid.ColumnDefinitions>  
    <Grid.RowDefinitions>  
        <RowDefinition Height="100"/>  
        <RowDefinition Height="*" x:Name="rowCanvas" />  
        <RowDefinition Height="100"/>  
    </Grid.RowDefinitions>  
    <TextBlock x:Name="designPage" Grid.Row="0" TextWrapping="Wrap" Text="Create icon" VerticalAlignment="Top" HorizontalAlignment="Left" FontSize="50" Height="91" Grid.RowSpan="2" Margin="50,10,0,0"/>  
    <Border Grid.Row="0" BorderBrush="Black" BorderThickness="0,2,0,2" Grid.ColumnSpan="2"/>  

    <StackPanel Grid.Row="2" Orientation="Horizontal" HorizontalAlignment="Left" Grid.ColumnSpan="2">  
            
          
        <Button x:Name="toRaster" Content="Raster" VerticalAlignment="Top" HorizontalAlignment="Left" FontSize="35" BorderBrush="#FF151515" Foreground="#FF0A0A0A"  
                         Background="#FF10EFEA"  BorderThickness="2,2,2,2" Width="150" Height="80" Margin="30,10,0,0" IsEnabled="True" Click="toRaster_Click" />  

    </StackPanel>  

    <Border Grid.Row="2" BorderBrush="Black" BorderThickness="0,2,0,2" Grid.ColumnSpan="2"/>  

    <Canvas x:Name="gridCanvas" Grid.Row="1" Background="White" Grid.ColumnSpan="2" />  

             
    <TextBlock x:Name="comments" Grid.Row="1" HorizontalAlignment="Left" VerticalAlignment="Top" Margin="10,210,0,0" TextWrapping="Wrap"/>  
  
</Grid>  

</Page>

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Windows.UI; // Defines Color

namespace IconCreator
{
public class Data
{

    // Test  
    public static int count = 0;  

    // Data entered on Setup Page  
    //===========================  
    public static bool setupDataDefined_bool = false;  


    // Data defining the grid  
     
    public static int horPix_int = 0;  
    public static string horPix_str;  

    public static int verPix_int = 0;  
    public static string verPix_str;  

    public static int x0Grid_int = 0;  
    public static int y0Grid_int = 0;  
    public static int gridCellSize_int = 0;  

    public static int[,] gridColoursRef_int;  
    public static int[,] gridColoursCur_int;  

    // Data defining the colours in use  
    public static int numColours_int;  
    public static Color[] colours;  
    public static int[,] xyColours_int;  
    public static int colourActive_int = 0;  

    // Data defining the size of the canvas  
    public static int canvasHeight_int;  
    public static int canvasWidth_int;  

    // Data entered on LoadIcon Page  
    //==============================  
    public static bool iconIsActive_bool = false;  
    public static bool iconDataDefined_bool = false;  

    // Data defining the icon loaded  
    public static int iconPixHor_int;  
    public static int iconPixVer_int;  
    public static int iconPixBits_int;  
    public static int iconFormat_int;  
    public static string iconLine_str;  

    // Data defining the decoded icon to bytes  
    public static byte[] iconBytes_byte;  
    public static int iconBytesHor_int;  
    public static int iconBytes_int = 0;  

    // Data entered on SaveIcon  
    //=========================  
    public static int saveBitsPerPix_int;  
    public static int saveColoursPerPixel_int;  

    // Icon position on grid  
    //======================  

    // Data definingg the position of the icon on the grid  
    public static bool iconPositionFixed_bool = false;  
    public static int iconPosX_int;  
    public static int iconPosY_int;  
    public static bool iconColourIsSet_bool = false;  

    // Text overlay on grid  
    //=====================  
    public static bool textIsActive_bool = false;  
    public static bool textDataDefined_bool = false;  
    public static bool textRasterDefined_bool = false;  

    public static int textFontSize_int;  
    public static string textText_str;  
    public static string textFontFamily_str;  
    public static int textGridCellHor_int;  
    public static int textGridCellVer_int;  
    public static int textGridDeltaPixHor_int;  
    public static int textGridDeltaPixVer_int;  
    public static bool textColourIsSet_bool = false;  
    public static string textStepSize_str;  

    public static void CreateColourArray(int num)  
    {  
        numColours_int = num;  
        colours = new Color[num];  
        xyColours_int = new int[num, 2];  
    }  

    public static void CreateGridColourArray()  
    {  
        gridColoursRef_int = new int[horPix_int, verPix_int];  
        gridColoursCur_int = new int[horPix_int, verPix_int];  

        for( int iVer = 0; iVer < verPix_int; iVer++ )  
        {  
            for( int iHor = 0; iHor < horPix_int; iHor++ )  
            {  
                gridColoursCur_int[iHor, iVer] = 0;  
                gridColoursRef_int[iHor, iVer] = 0;  
            }  
        }  
          
    }  

    public static void CreateIconByteArray(int num)  
    {  
        iconBytes_byte = new byte[num];  
    }  
}  

}

Please, run the code and you should see an "S" in red on a 10x15 grid in green. In addition to the "S" there is one grid cell in red.

Please, then click on the button "Raster". The grid cell in red is there, but the S has disappeared. It should have been converted to filling cells.

I believe that the problem is timing. The reading of the pixels from the screen is an async process. (Can I avoid making it async?). While that is running the main thread continues and re-fills the 10x15 grid. It does that so fast that when the screen is read, it actually reads the re-filled 10x15 grid. The re-fill is supposed to show the result of the reading of the screen and the conversion to cell raster. Somehow, I need to block the main thread until the ImageOnRaster() function has completed.

Sorry, but I am not that experienced using C#. I need to better understand async processes and synchronization. I believe I have to use await for these calls:

await renderTargetBitmap.RenderAsync(this.gridCanvas);
var renderBitmapPixels = await renderTargetBitmap.GetPixelsAsync(); // return an IBuffer stream

to get the compiler to accept the code. Hence the function ImageOnRaster() has to be async, and that is the problem, I think.

Please, advice on the best way to solve this timing issue.

Thank you for your interest and effort.

Regards,
Jorgen