Help with Windows Named Pipes

Question

Help with Windows Named Pipes

Ken Gibson 0

Win32 experts,

I'm writing an app that includes a master process written in C# that forks a subprocess written in Python and compiled into an exe. I am attempting to open named pipe between the two and despite working through all online tips and all of copilot's suggestions for getting this to work, am still getting 'The system cannot find the file specified.' error in the child process.

I have:

Used the correct pipe naming format. In this case: "\.\pipe\derPipe"
Confirmed the child process receives and uses the correct pipe name;
In the parent, confirmed the pipe is successfully created;
In the parent, insure the child is authorized to open the pipe using copilot's suggested code: PipeSecurity pipeSecurity = new PipeSecurity(); pipeSecurity.AddAccessRule(new PipeAccessRule(new SecurityIdentifier(WellKnownSidType.WorldSid, null), PipeAccessRights.FullControl, AccessControlType.Allow));
I then pass this PipeSecurity object in the call to create the NamedPipeServerStream;
In the Python child, wait for the pipe to be ready with: win32pipe.WaitNamedPipe(pipeName, 2000) - after confirming pipeName contains the correct name. This function is where I actually get the 'system cannot find the file specified' error.

At this point I've run out of ideas, online suggestions, & copilot suggestions for getting this to work. Any additional tips are greatly appreciated.

Thanks!

Viorel 122.5K Reputation points

2024-11-13T15:54:12.5366667+00:00

Did you represent correctly the ‘\’, such as @"\.\pipe\derPipe" in C#?. Probably the name should be @"\\.\pipe\derPipe".'

By the way, instead of additional pipes, you can use the default streams offered by Process class. The C# will use WriteLine. The Python will use sys.stdin.read().
Ken Gibson 0 Reputation points

2024-11-13T16:49:37.5466667+00:00

Thanks for the suggestion but yes, I am already using @"\.\pipe\derPipe" and am confirming the Python child process does receive the correct pipe name with the correct \ characters.

I forgot the mention that I am also confirming the parent and child open the pipe with the same access modes: PipeDirection.InOut when creating the pipe in the parent and win32file.GENERIC_READ | win32file.GENERIC_WRITE when creating the file in the Python child.

I will look into your suggestion to use the default streams instead. Thank you.
Ken Gibson 0 Reputation points

2024-11-13T16:58:48.43+00:00

Yes I did prefix the pipe name with @ and confirmed the client receives the correct \ characters.

I forgot to add that I also made sure the parent and child open the pipe with the same access modes:

PipeDirection.InOut when creating the NamedPipeServer in the parent, and win32file.GENERIC_READ | win32file.GENERIC_WRITE when creating the file in the Python child.

I will look into using the default streams instead. Thanks for the suggestion.
Darran Rowe 1,986 Reputation points

2024-11-13T17:21:36.0466667+00:00

I believe you may have missed what Viorel stated.

If you look at the parameter list for CreateNamedPipe, it states that the pipe name must have the form \\.\pipe\pipename. In other words, the documentation states that the string must start with two \ characters. If you are writing out two \ charactes then the post is eating one of them, if you are not writing out two \ characters then be aware that the amount of \ characters is important.

The \\.\ prefix is used to access the Windows device namespace, it is also the prefix for UNC paths targetting the local machine.
Ken Gibson 0 Reputation points

2024-11-13T17:56:59.9+00:00

Yes, it seems one backslash is getting lost in this post. I AM prefixing the pipe name with two backslashes and confirming it is received in the child process prefixed with both backslashes.

Thanks,

Ken
Ken Gibson 0 Reputation points

2024-11-13T19:25:12.0966667+00:00

It does seem that when I post to this forum, some backslashes get lost. To confirm, I am including the correct number of backslashes in my call to win32pipe.WaitNamedPipe(). As an experiment, I doubled the backslashes (meaning the name started with four backslashes) and WaitNamedPipe returned an error saying the pathname is invalid, instead of the 'system cannot find the file specified' error I was previously getting which seems to confirm I am passing it a valid pipe name.I also tried doing an 'ls' from Powershell on backslash-backslash-dot-backslash-pipe while my master process was running and after it successfully created the pipe. The 'ls' returned "cannot find path [the pipe folder] because it does not exist". Not sure if that is expected behavior.

Thanks for all the suggestions. In parallel I have been continuing to query Copilot in my VS code project and it just keeps confirming that my code is correct.
Darran Rowe 1,986 Reputation points

2024-11-13T21:45:09.4833333+00:00
Just to ask the stupidly obvious question, but are there any potential lifetime issues here? Could you be releasing the C# reference too early? Doing a quick test of this using a C++ application does show that the pipe was created without issue.

#include <Windows.h> #include <conio.h> int wmain() { HANDLE pipe = CreateNamedPipeW(LR"(\\.\pipe\testpipe)", PIPE_ACCESS_DUPLEX, 0, PIPE_UNLIMITED_INSTANCES, 1024, 1024, NMPWAIT_USE_DEFAULT_WAIT, nullptr); while (!_kbhit()) { if (_getwch() == L'x') { break; } } CloseHandle(pipe); pipe = nullptr; return 0; }

Then using get-childitem \\.\pipe\ in powershell shows:

Other applications are able to open this without issue. The individual pieces look like they are working here at the fundamental level.

If you keep having issues, then I really suggest writing a small sample application showing exactly what the applications are doing. If something is going wrong, then it is possible that nobody will be able to pick it out from a description alone.
RLWA32 49,536 Reputation points

2024-11-13T22:24:57.1666667+00:00

Is the parent process creating the pipe before or after it starts the child process?
Ken Gibson 0 Reputation points

2024-11-14T01:02:12.44+00:00

The pipe is created first.
Ken Gibson 0 Reputation points

2024-11-14T02:16:08.93+00:00

It seems I'm not authorized to reply with code snipets for whatever reason.

But executing your get-childitem command shows that the path \.\pipe does not exist even though creating a NamedPipeServerStream succeeds (and the pipe has not been closed).
Darran Rowe 1,986 Reputation points

2024-11-14T10:49:45.6833333+00:00

Just to ask, was the format of the path exact?

The path in the screenshot is the exact path that I used in order to list the pipes. The trailing backslash is required in this case.To go into a little unnecessary detail, there is a translation that goes on between the names that are used in the Windows API and what Windows sees internally. As a little example, when C:\ is used, this is translated to the path \??\C:\.

Interestingly, the special names COM1, CON, PRN and more are in here. But generally, the Windows API namespace doesn't have general access to this internal Windows namespace. This is documented in Naming Files, Paths, and Namespaces under the Win32 Device Namespaces.

Pipe is a device in this namespace, but it isn't a special name.

This means that the device namespace path must be used here. Since the desire is to also query the contents of the pipe device, then the path needs to reflect that. It is like when you wish to access the contents of a drive's root directory, it must end in a backslash.
Ken Gibson 0 Reputation points

2024-11-14T17:35:59.7066667+00:00
Thanks for the detail and yes, with the trailing backslash I see the following (I executed this while my master process is running and has created the named pipe but has not closed it)

New-Item -Path \.\pipe\ -ItemType Directory

New-Item : An item with the specified name \.\pipe\ already exists.

At line:1 char:1

New-Item -Path \.\pipe\ -ItemType Directory

+ CategoryInfo : ResourceExists: (\\.\pipe\:String) [New-Item], IOException + FullyQualifiedErrorId : DirectoryExist,Microsoft.PowerShell.Commands.NewItemCommand
Ken Gibson 0 Reputation points

2024-11-14T17:45:04.0866667+00:00

A question: In C, you can check if CreateNamedPipeA() returns INVALID_HANDLE_VALUE.

How do you do something similar in C# to confirm that a new NamedPipeServerStream is valid? In my case, the constructor does not raise any exceptions but I would like to add an additional check to confirm the new NamedPipeServerStream object was successfully created.

Thanks
RLWA32 49,536 Reputation points

2024-11-14T20:01:09.9166667+00:00

You should be able to check the validity of the created object by retrieving and interrogating its SafePipeHandle property.
Ken Gibson 0 Reputation points

2024-11-14T21:34:41.0433333+00:00

Thank you all for your help and timely responses. I'm building a Proof-of-Concept for my client to show they can integrate their existing C# and Python into one application. After two days of beating my head against this named pipe problem without success, I'm going to proceed with using Streams to the child process's stdin and stdout. That is working fine and will be sufficient for the PoC.

Thanks again for the help.

Your answer

Viorel 122.5K Reputation points

2024-11-13T15:54:12.5366667+00:00

Did you represent correctly the ‘\’, such as @"\.\pipe\derPipe" in C#?. Probably the name should be @"\\.\pipe\derPipe".'

By the way, instead of additional pipes, you can use the default streams offered by Process class. The C# will use WriteLine. The Python will use sys.stdin.read().
Ken Gibson 0 Reputation points

2024-11-13T16:49:37.5466667+00:00

Thanks for the suggestion but yes, I am already using @"\.\pipe\derPipe" and am confirming the Python child process does receive the correct pipe name with the correct \ characters.

I forgot the mention that I am also confirming the parent and child open the pipe with the same access modes: PipeDirection.InOut when creating the pipe in the parent and win32file.GENERIC_READ | win32file.GENERIC_WRITE when creating the file in the Python child.

I will look into your suggestion to use the default streams instead. Thank you.
Ken Gibson 0 Reputation points

2024-11-13T16:58:48.43+00:00

Yes I did prefix the pipe name with @ and confirmed the client receives the correct \ characters.

I forgot to add that I also made sure the parent and child open the pipe with the same access modes:

PipeDirection.InOut when creating the NamedPipeServer in the parent, and win32file.GENERIC_READ | win32file.GENERIC_WRITE when creating the file in the Python child.

I will look into using the default streams instead. Thanks for the suggestion.
Darran Rowe 1,986 Reputation points

2024-11-13T17:21:36.0466667+00:00

I believe you may have missed what Viorel stated.

If you look at the parameter list for CreateNamedPipe, it states that the pipe name must have the form \\.\pipe\pipename. In other words, the documentation states that the string must start with two \ characters. If you are writing out two \ charactes then the post is eating one of them, if you are not writing out two \ characters then be aware that the amount of \ characters is important.

The \\.\ prefix is used to access the Windows device namespace, it is also the prefix for UNC paths targetting the local machine.
Ken Gibson 0 Reputation points

2024-11-13T17:56:59.9+00:00

Yes, it seems one backslash is getting lost in this post. I AM prefixing the pipe name with two backslashes and confirming it is received in the child process prefixed with both backslashes.

Thanks,

Ken
Ken Gibson 0 Reputation points

2024-11-13T19:25:12.0966667+00:00

It does seem that when I post to this forum, some backslashes get lost. To confirm, I am including the correct number of backslashes in my call to win32pipe.WaitNamedPipe(). As an experiment, I doubled the backslashes (meaning the name started with four backslashes) and WaitNamedPipe returned an error saying the pathname is invalid, instead of the 'system cannot find the file specified' error I was previously getting which seems to confirm I am passing it a valid pipe name.I also tried doing an 'ls' from Powershell on backslash-backslash-dot-backslash-pipe while my master process was running and after it successfully created the pipe. The 'ls' returned "cannot find path [the pipe folder] because it does not exist". Not sure if that is expected behavior.

Thanks for all the suggestions. In parallel I have been continuing to query Copilot in my VS code project and it just keeps confirming that my code is correct.
Darran Rowe 1,986 Reputation points

2024-11-13T21:45:09.4833333+00:00

Just to ask the stupidly obvious question, but are there any potential lifetime issues here? Could you be releasing the C# reference too early? Doing a quick test of this using a C++ application does show that the pipe was created without issue.

#include <Windows.h> #include <conio.h> int wmain() { HANDLE pipe = CreateNamedPipeW(LR"(\\.\pipe\testpipe)", PIPE_ACCESS_DUPLEX, 0, PIPE_UNLIMITED_INSTANCES, 1024, 1024, NMPWAIT_USE_DEFAULT_WAIT, nullptr); while (!_kbhit()) { if (_getwch() == L'x') { break; } } CloseHandle(pipe); pipe = nullptr; return 0; }

Then using get-childitem \\.\pipe\ in powershell shows:

Other applications are able to open this without issue. The individual pieces look like they are working here at the fundamental level.

If you keep having issues, then I really suggest writing a small sample application showing exactly what the applications are doing. If something is going wrong, then it is possible that nobody will be able to pick it out from a description alone.
RLWA32 49,536 Reputation points

2024-11-13T22:24:57.1666667+00:00

Is the parent process creating the pipe before or after it starts the child process?
Ken Gibson 0 Reputation points

2024-11-14T01:02:12.44+00:00

The pipe is created first.
Ken Gibson 0 Reputation points

2024-11-14T02:16:08.93+00:00

It seems I'm not authorized to reply with code snipets for whatever reason.

But executing your get-childitem command shows that the path \.\pipe does not exist even though creating a NamedPipeServerStream succeeds (and the pipe has not been closed).
Darran Rowe 1,986 Reputation points

2024-11-14T10:49:45.6833333+00:00

Just to ask, was the format of the path exact?

The path in the screenshot is the exact path that I used in order to list the pipes. The trailing backslash is required in this case.To go into a little unnecessary detail, there is a translation that goes on between the names that are used in the Windows API and what Windows sees internally. As a little example, when C:\ is used, this is translated to the path \??\C:\.

Interestingly, the special names COM1, CON, PRN and more are in here. But generally, the Windows API namespace doesn't have general access to this internal Windows namespace. This is documented in Naming Files, Paths, and Namespaces under the Win32 Device Namespaces.

Pipe is a device in this namespace, but it isn't a special name.

This means that the device namespace path must be used here. Since the desire is to also query the contents of the pipe device, then the path needs to reflect that. It is like when you wish to access the contents of a drive's root directory, it must end in a backslash.
Ken Gibson 0 Reputation points

2024-11-14T17:35:59.7066667+00:00

Thanks for the detail and yes, with the trailing backslash I see the following (I executed this while my master process is running and has created the named pipe but has not closed it)

New-Item -Path \.\pipe\ -ItemType Directory

New-Item : An item with the specified name \.\pipe\ already exists.

At line:1 char:1

New-Item -Path \.\pipe\ -ItemType Directory

+ CategoryInfo : ResourceExists: (\\.\pipe\:String) [New-Item], IOException + FullyQualifiedErrorId : DirectoryExist,Microsoft.PowerShell.Commands.NewItemCommand
Ken Gibson 0 Reputation points

2024-11-14T17:45:04.0866667+00:00

A question: In C, you can check if CreateNamedPipeA() returns INVALID_HANDLE_VALUE.

How do you do something similar in C# to confirm that a new NamedPipeServerStream is valid? In my case, the constructor does not raise any exceptions but I would like to add an additional check to confirm the new NamedPipeServerStream object was successfully created.

Thanks
RLWA32 49,536 Reputation points

2024-11-14T20:01:09.9166667+00:00

You should be able to check the validity of the created object by retrieving and interrogating its SafePipeHandle property.
Ken Gibson 0 Reputation points

2024-11-14T21:34:41.0433333+00:00

Thank you all for your help and timely responses. I'm building a Proof-of-Concept for my client to show they can integrate their existing C# and Python into one application. After two days of beating my head against this named pipe problem without success, I'm going to proceed with using Streams to the child process's stdin and stdout. That is working fine and will be sufficient for the PoC.

Thanks again for the help.

Share via

Help with Windows Named Pipes

Your answer