Hi and thanks for reaching out. My name is William. I'm a Windows technical expert. I'll be happy to help you out today.
Setting in HKEY_LOCAL_MACIHNE applies to all users. To limit scope, you will need to set specifically for your child by logging into their account and modifying the registry under HKCU.
Now, before you can do this, you need to understand that policies key under HKCU is protected and only admin can edit those in registry. However, you cannot simply right-click and choose to run regedit.exe as an administrator because once you specify your admin account, the security context switches to the administrative user and any changes you make in HKCU will apply to your admin account.
To work around that, you will need to temporarily promote your child to admin account > logon as your child > make the regedit changes below > then remove admin role from your child account.
For IE: HKEY_CURRENT_USER\Software\Policies\Microsoft\Internet Explorer\Privacy
DWORD EnableInPrivateBrowsing = 0
For Chrome: HKEY_CURRENT_USER\SOFTWARE\Policies\Google\Chrome
DWORD IncognitoModeAvailability = 1
For new Edge: HKEY_CURRENT_USER\SOFTWARE\Policies\Microsoft\Edge
DWORD InPrivateModeAvailability = 1
For Firefox: HKEY_CURRENT_USER\SOFTWARE\Policies\Mozilla\Firefox
DWORD DisablePrivateBrowsing = 1
I agree, it is convoluted.