WPF 3D to 2D projection appears wrong?

WoodManEXP 1

Have a case where a point on the PerspectiveCamera’s LookDirection vector does not project to the center of the Viewport3D. Any thoughts as to why?

Here are details of the case (all the values are rounded for brevity): The Viewport3D is 778, 405. Any visible Point3D the camera is looking directly at would project to (389, 202.5). The PerspectiveCamera is located at (1379.4, -577.7, 0.7).

In the first case the center of the White sphere is (13,080.3,-6,491.9,-407.9) which is 13,117 units from the camera. Camera’s LookDirection is (0.89, -0.45, -0.03). The White sphere projects to the visual center of the Viewport3D. MathUtils.TryWorldToViewportTransform reports the sphere’s center point projects to (389, 202.5), as expected. The black lines are added to show the center of the Viewport3D.

White sphere

In the second case the center of the Blue sphere is (1,383.0,-579.6, 0.3) which is 4.1 units from the camera. Camera’s LookDirection is (0.88,-0.46,-0.11). The Blue sphere does NOT render to the visual center of the Viewport3D. MathUtils.TryWorldToViewportTransform reports the center point being projected to (389, 202.5), as expected. Visually though the blue sphere is not drawing at center, it is at more like (295, 303).

Blue sphere

Placing the camera at the 4.1 distance from the White sphere it projects correctly to the center of the Viewport3D. The difference is that the White sphere is further from the (0, 0, 0) origin than the Blue sphere.

The Blue sphere will project more correctly to the center of the Viewport3D if the camera is “far away” from it, like out near the White sphere.

The Blue sphere is mathematically positioned to be at the center of the Viewport3D. Any thoughts as to why the Blue sphere is not rendering at the center of the Viewport3D in the second case above?

Thank you!

Hui Liu-MSFT 47,341 Reputation points Microsoft Vendor

2023-09-12T08:15:38.72+00:00

Hi,@WoodManEXP.Welcome Microsoft Q&A.

The behavior you're observing is due to the nature of perspective projection in 3D graphics, where objects closer to the camera may not appear at the center of the screen even if their projected 2D position is calculated as such. The issue is likely due to the interaction of the camera's perspective projection and the depth of the objects.

Depth: As you mentioned, the White sphere is further from the origin (0,0,0) than the Blue sphere. Objects closer to the origin will project differently in perspective projection, potentially leading to differences in their apparent positions.

Field of View: The camera's field of view angle can affect how much distortion occurs. A wider field of view will cause more distortion, especially for objects at different depths.

Aspect Ratio: The aspect ratio of the viewport can also affect how objects appear on the screen.

To achieve consistent visual alignment of objects in 3D space with the center of the 2D screen, you might need to consider adjusting the position or scaling of objects based on their depth or using techniques like orthographic projection, which doesn't have the same distortion effect for objects at different depths.
WoodManEXP 1 Reputation point

2023-09-12T13:20:12.3066667+00:00

Hi Hui Liu,

TY for the reply. Follow-up question please:

In the Blue sphere case, incremental adjustments to the PrespecticeCamera’s LookDirection will bring the sphere to the center of the 2D viewport. But that LookDirection is now offset and no longer mathematically points to the sphere center in the 3D space. I had tried to use MathUtils.TryWorldToViewportTransform to calculate that offset to apply a correction. It must not include the distortion in its calculations. Is there a way to calculate the 3D to 2D projection that includes the perspective distortion?

2 answers

Hui Liu-MSFT 47,341 Reputation points Microsoft Vendor

2023-09-13T02:05:27.4866667+00:00

Hi,@WoodManEXP. To calculate a 3D-to-2D projection that includes the perspective distortion, you could try to perform a perspective projection transformation.

You could try to see if the solutions here and projection-3d-2d are helpful to you.

If the response is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
Please sign in to rate this answer.
WoodManEXP 1 Reputation point

2023-09-17T15:47:10.2333333+00:00

It seems like any object centered on the LookDirection vector would appear in the middle of the 2D viewport, irrespective of the WPF 3D projection type.

Attached is a short VS project that positions a Blue and a Red sphere centered on the Perspective Camera's LookDirection vector. One sphere is centered and the other is not:

Pls check out the short C# project, at the link below, and help me understand why the Red sphere is not centered in the 2D viewport. Unzip the project and open LookDirection Test.sln to open the project in Visual Studio.

https://drive.google.com/file/d/1R_qqZT4jEMMt_Z0e8fvhfyZXEGXsgiTk/view?usp=sharing

TY!

WoodManEXP 1 Reputation point

2023-09-18T17:41:43.56+00:00

Hi, Hui Liu,

Using https://stackoverflow.com/questions/519106/projecting-a-3d-point-to-a-2d-screen-coordinate both the MathUtils.TryWorldToViewportTransform and Visual3D.TransformToAncestor produce the same result. Neither returns the viewport pixel value where WPF renders the object, in this situation.

Please see follow-on question https://learn.microsoft.com/en-us/answers/questions/1371696/wpf-3d-to-2d-projection-on-lookdirection-vector-ap.

Is there a way in WPF to perform the 3D to 2D projection that will produce the 2D viewport pixel coordinates where WPF will render the 3D point in 2D space?

TY!

Hui Liu-MSFT 47,341 Reputation points Microsoft Vendor

2023-09-19T06:51:58.8933333+00:00

Hi,@WoodManEXP. I have tested the project. And I'm researching and looking for solutions, will come back if there are any updates.

WoodManEXP 1 Reputation point

2023-09-19T17:52:08.7433333+00:00

This may be the result of rounding conditions deeper in WPF when it converts to the 2D viewport pixel coordinates.

Is there a way in WPF to perform the 3D to 2D projection that will produce the 2D viewport pixel coordinates where WPF will render the 3D point in 2D space? If it were possible to do this then work-arounds could be implemented at higher levels in the application when these cases occur.

I been looking through the .Net reference source, https://referencesource.microsoft.com/and https://github.com/dotnet/wpf/tree/main/src/Microsoft.DotNet.Wpfto try to locate where such calculations take place and see how they are accomplished. No luck thus far...

TY!

WoodManEXP 1 Reputation point

2023-09-22T19:13:56.0733333+00:00

Here are additional notes from digging deeper:

In the above image the Blue sphere’s center is 3.87 units from the Camera and is mathematically directly on the Camera’s LookDirection vector (but is not rendering in viewport center).

Blue sphere located at (1382.8933529409558,-580.0058610620723,0.345352054033831)
Camera located at (1379.4825173770382,-578.2092937590097,0.7700351819921936)
LookDirection (0.8794491766028759,-0.46322656304640114,-0.10950021488902285)

Now, rotate the camera so that the Blue sphere is positioned roughly in viewport center:

Camera position unchanged, but it's LookDirection is now

LookDirection (0.8700569211410617,-0.4805890656032783,-0.10970462158498284)

Camera no longer mathematically looking at the Blue sphere, but the sphere is in the middle of the viewport. The angle between the camera's LookDirection and direction to the sphere is 7.066574389261597 degrees.

Now move the camera along its current LookDirection toward the Blue sphere till it is closer; in this case a distance of 0.7580609447501166 to the sphere’s center.

The angle between the camera's LookDirection and direction to the sphere is now 46.52316217884858 degrees. (That makes sense because mathematically the camera is passing the sphere to one side rather than moving towards it) The sphere continues rendering in the viewport center but is mathematically ~47 degrees off the Camera’s LookDirection vector.

There must be rounding taking place somewhere in the transformation chain to get from 3D to 2D viewport pixel coordinates.

I have been searching through the .Net reference source (https://referencesource.microsoft.com/). so far unsuccessfully, to understand what these calculations are and where they take place. If it were possible to recreate or get access to the 2D pixel values, it should be possible to make adjustments to the Camera’s direction vectors to compensate.

Does anyone have knowledge of what is happening with the PerpectiveCamera and its 3D to 2D pixel coordinate transformations?

TY!
Sign in to comment
WoodManEXP 1 Reputation point

2023-11-26T22:43:22.4666667+00:00

After lots of research into WPF’s 3D capabilities (using the Viewport3D control to render 3D) including reverse engineering parts of the WPF Viewport3D source code, the best way forward has been to give up using WPF’s 3D subsystem. It does not offer a usable means of dealing with or controlling, among others, the 3D to 2D projection problems baked into it.

Fortunately, a WPF OpenGL C# binding is available, it is called OpenTK and can be nicely woven into the WPF framework. Can be found at http://opentk.net.

OpenGL is somewhat of a paradigm shift from WPF’s 3D. It, at first, feels more tedious to learn and use. Once getting into the Zen of OpenGL though it is, IMHO, a nice improvement. It provides excellent control of the many things that are hard to control in WPF 3D, including the 3D to 2D projection issues.

OpenGL seems to offer better performance in terms of both processing and memory utilization.

WPF 3D’s hit testing will be missed though. It is baked more into WPF 3D’s object model, not the case with OpenGL…
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Share via

WPF 3D to 2D projection appears wrong?

2 answers