Is anybody familiar with the error "A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetworkConfig"

Question

Is anybody familiar with the error "A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetworkConfig"

Josh Gardner 1

I am attempting to configure my testbed environment's DAG's network, but any DAG network command I run gives me errors. Please note, my testbed (and production) is Exchange 2016 on-prem (CU22 with Jan SU installed).

I can find NOTHING online pertaining to these errors at all. Nothing shows in my event viewer (application or system). My testbed has NO HBSS or A/V on it, and it only has 1 network. I wonder if there was some sort of Exchange CU/SU or .NET or even an underlying AD issue (DNS?) that is causing any DAG network command to fail. I've tried these commands from all my Exchange servers (both my test bed and my production), specifically from the Primary Active Manager - to no avail. I am simply just unable to configure the DAG networks.

Figure 1 Test Bed:

Figure 2 Production (because we have "ReplicationNetworks" due iSCSI storage arrays):

Figure 3 Event Log

Errors in Text format:
A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetworkConfig
A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetwork

Does anybody have any familiarity with these 2 errors? If so... please tell me how to get passed them.

Joyce Shen - MSFT 16,541 Reputation points

2022-02-03T02:17:47.1+00:00
Hi @Josh Gardner

I see some related posts shared the issue caused by networking, you could temporarily disable firewall to see if any help. In addition, please also try adding Full Permissions to the Exchange Trusted Subsystem onto the CNO of the DAG, just like below:

And what's the current configuration for your Exchange dag? Note to clear your personal information in the result.

Get-DatabaseAvailabilityGroupNetwork <DAGNetworkName> | Format-List Get-DatabaseAvailabilityGroup | Format-List

And here is the official document introduces about
Create a database availability group network in Exchange Server
Josh Gardner 1 Reputation point

2022-02-03T14:09:23.927+00:00

I appreciate your suggestions @Joyce Shen - MSFT , but I have an IP-less DAG in both my testbed and my production environment, so there is no CNO. I have also confirmed that Windows Firewall is turned off in both environments.

The DAG FL is rather long so here is a two part picture

DAG network FL in my testbed

DAG network FL in production - hope my explanations are not too convoluted

Honestly I don't know how long this may have been going on, I am aware that DAG Network commands are rather uncommon after initial configuration. Its a set it and forget it. For reference, I have looked all over the Internet to see if there is something I'm missing, its not that something is horribly misconfigured, its the fact that I cannot configure the DAG networks at all.

Also it looks like the images I uploaded here with command results can no longer be loaded on the page.
Josh Gardner 1 Reputation point

2022-02-03T14:21:09.12+00:00
To be super clear, I cannot even configure the networks that I DO want. The goal here is to:

remove the accidentally created ipv6 DAG network (unable)

disable replication on the second and third iSCSI networks to match same network configuration of the first iSCSI network settings with Set-DatabaseAvailabilityGroupNetwork -ReplicationEnabled $False (unable)

ignore network on the second and third iSCSI networks to match same network configuration of the first iSCSI network settings with Set-DatabaseAvailabilityGroupNetwork -IgnoreNetwork $True (unable)

"rescan" the DAG network with Set-DatabaseAvailabilityGroup -DiscoverNetworks (unable)

"reset" the manual DAG network configuration with Set-DatabaseAvailabilityGroup -ManualDagNetworkConfiguration $True (unable)

Something is preventing me from doing any sort of DAG network related commands. All of the above commands results in either of the below errors.
A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetworkConfig
A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetwork
Josh Gardner 1 Reputation point

2022-02-03T14:31:45.47+00:00

To be even more super clear - my testbed environment is very simple in its design, and again, it does not have Windows Firewall, or any sort of AV on it. It only has one DAG network, and only 2 nodes. None of the DAG network commands I run in the testbed are successful, so it is in the same state as my production environment. Which leads me to believe it is not anything related specifically to the slightly more complex production environment.
Joyce Shen - MSFT 16,541 Reputation points

2022-02-08T08:22:52.203+00:00

Hi,

Have you checked if all services are running and the service health after installing the SU like this article introduces
Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.

And if you consider the issue is related to CU22 Jan SU, we could wait for the SU of Feb, it will publish soon,
Josh Gardner 1 Reputation point

2022-02-08T17:38:08.08+00:00

@Joyce Shen - MSFT Again, I appreciate your suggestions. All services are up, servers have been rebooted a few times just to be sure.

I went ahead and uninstalled Jan 2022 SU (KB5008631) and rebooted in my testbed, which did not remedy any of the errors
I went ahead and uninstalled Nov 2021 SU (KB5007409) and rebooted in my testbed, which did not remedy any of the errors

I suppose I could continue to roll back SU installations all the way back (due to the above testing, i'm on KB5007012 in my testbed), but I really dont know if there is a point. I haven't seen any errors in the event log that would indicate these DAG network errors are related to any Security Update with Exchange 2016 with CU22 installed. There is just.... nothing.

Is it somehow related to the CU?
Is it somehow related an older SU?
Is it somehow related to AD?
Is it somehow related to IIS?
Is it somehow related to .NET?
Joyce Shen - MSFT 16,541 Reputation points

2022-02-11T07:50:32.15+00:00

Hi,

Could you please try creating a new administrator account and make sure right permissions are granted to it then verify the result again?

And can you successfully create new DAG by running the New-DatabaseAvailabilityGroup command?
Josh Gardner 1 Reputation point

2022-02-16T19:09:40.347+00:00

@Joyce Shen - MSFT I'm pretty confident this error has nothing to do with permissions. I am in the Organization Management security group, and to my knowledge, I have zero issues running any other Exchange cmdlets beyond the DAG Network cmdlets

I was able to create a new IP-less DAG in my testbed (see picture 1). I did not add any servers to it HOWEVER, I was able to run at least ONE of the commands that I had not been able to before (see picture 2). What can explain this behavior please?
Josh Gardner 1 Reputation point

2022-02-16T20:34:44.263+00:00

As a follow up, I decided to back out of the original DAG in my testbed (TESTDAG). Since I was able to run the Set-DatabaseAvailabilityGroup -ManualDagNetworkConfiguration $True command on the new DAG in my testbed (TESTDAG2). I figured I'd remove my mailbox servers from the old and add them to the new. I also added a NIC to my testbed servers so that I can ensure that I can run DAG Network cmdlets. Sure enough, as soon as I added my first mailbox server into the new DAG and tried to run a DAG Network cmdlet it failed. Hopes have been dashed.
Joyce Shen - MSFT 16,541 Reputation points

2022-02-21T09:02:08.593+00:00

Hi @Josh Gardner

I would recommend you open a support ticket with Microsoft to solve this issue.

Global Customer service Phone number:
https://support.microsoft.com/en-us/topic/global-customer-service-phone-numbers-c0389ade-5640-e588-8b0e-28de8afeb3f2

If you have contract of paid support for on-premises Microsoft products and service, submit the ticket from this portal:
https://support.microsoft.com/en-us/gp/contactus81?audience=commercial
JBerg712 6 Reputation points

2022-02-22T20:57:33.817+00:00

Josh,

We have the exact same issue. We are migrating from Exchange 2013 to Exchange 2019 on the latest CU. For whatever reason, we cannot configure the DAG network because of the same error. It's very odd that the only error in the event logs is the cmdlet error, but states no reason as to why or what this is referring to exactly. I've turned up my event logging to EXPERT level and it's not recording anything when you attempt a network dag configuration change.

I'm working on it as well, so if I find anything out, i'll report back here.

JB
Josh Gardner 1 Reputation point

2022-02-23T17:44:53.71+00:00

@JBerg712 Thanks so much. My IT Leadership is trying to convert some stuff into eligible Microsoft Technical Support credits. I'm kind of stuck at the moment in that regard. I tested yet a 3rd Exchange environment that we have, and that one also gave me the same error. To sum up, all 3 Exchange environments that I manage cannot run Set-DatabaseAvailabilityGroupNetwork commands at all. Its very frustrating.

I will say, I am in the DoD so that "may" come into play because we harden things in a very particular way, but its all documented and I've used my testbed Exchange environment quite thoroughly to try and essentially create a "blank slate" Exchange enterprise environment (literally zero GPOs applied to the Exchange Servers, no HBSS/Firewalls, etc) and I still run into the error. At first I thought it may be a due to a hardening setting, but now I'm really thinking it may be a bug in the CU (not the SU, because I even uninstalled those in the testbed). I would GREATLY appreciate an update for any resolution if you are able to find one. Thanks again for posting.
Adnan Abdul Salam 96 Reputation points

2022-03-07T13:09:37.693+00:00

i am facing the same issue , not able to run the command or configure from GUI , i need to disable replication on MAPI network but below error comes either i do from GUI or Cmdlet

A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetwork

i have Jan SU installed ,

Firewall is disabled. no antivirus firewall etc.
Josh Gardner 1 Reputation point

2022-03-09T16:00:09.267+00:00

@Adnan Abdul Salam Thanks for posting, I'm hoping more and more people will comment. Maybe this will gain visibility and traction with the Microsoft engineers and they can dive into it. More people reporting an issue makes me think there is a bug somewhere.

Joyce Shen - MSFT 16,541 Reputation points

2022-02-03T02:17:47.1+00:00

Hi @Josh Gardner

I see some related posts shared the issue caused by networking, you could temporarily disable firewall to see if any help. In addition, please also try adding Full Permissions to the Exchange Trusted Subsystem onto the CNO of the DAG, just like below:

And what's the current configuration for your Exchange dag? Note to clear your personal information in the result.

Get-DatabaseAvailabilityGroupNetwork <DAGNetworkName> | Format-List Get-DatabaseAvailabilityGroup | Format-List

And here is the official document introduces about
Create a database availability group network in Exchange Server
Josh Gardner 1 Reputation point

2022-02-03T14:09:23.927+00:00

I appreciate your suggestions @Joyce Shen - MSFT , but I have an IP-less DAG in both my testbed and my production environment, so there is no CNO. I have also confirmed that Windows Firewall is turned off in both environments.

The DAG FL is rather long so here is a two part picture

DAG network FL in my testbed

DAG network FL in production - hope my explanations are not too convoluted

Honestly I don't know how long this may have been going on, I am aware that DAG Network commands are rather uncommon after initial configuration. Its a set it and forget it. For reference, I have looked all over the Internet to see if there is something I'm missing, its not that something is horribly misconfigured, its the fact that I cannot configure the DAG networks at all.

Also it looks like the images I uploaded here with command results can no longer be loaded on the page.
Josh Gardner 1 Reputation point

2022-02-03T14:21:09.12+00:00

To be super clear, I cannot even configure the networks that I DO want. The goal here is to:

remove the accidentally created ipv6 DAG network (unable)

disable replication on the second and third iSCSI networks to match same network configuration of the first iSCSI network settings with Set-DatabaseAvailabilityGroupNetwork -ReplicationEnabled $False (unable)

ignore network on the second and third iSCSI networks to match same network configuration of the first iSCSI network settings with Set-DatabaseAvailabilityGroupNetwork -IgnoreNetwork $True (unable)

"rescan" the DAG network with Set-DatabaseAvailabilityGroup -DiscoverNetworks (unable)

"reset" the manual DAG network configuration with Set-DatabaseAvailabilityGroup -ManualDagNetworkConfiguration $True (unable)

Something is preventing me from doing any sort of DAG network related commands. All of the above commands results in either of the below errors.
A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetworkConfig
A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetwork
Josh Gardner 1 Reputation point

2022-02-03T14:31:45.47+00:00

To be even more super clear - my testbed environment is very simple in its design, and again, it does not have Windows Firewall, or any sort of AV on it. It only has one DAG network, and only 2 nodes. None of the DAG network commands I run in the testbed are successful, so it is in the same state as my production environment. Which leads me to believe it is not anything related specifically to the slightly more complex production environment.
Joyce Shen - MSFT 16,541 Reputation points

2022-02-08T08:22:52.203+00:00

Hi,

Have you checked if all services are running and the service health after installing the SU like this article introduces
Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.

And if you consider the issue is related to CU22 Jan SU, we could wait for the SU of Feb, it will publish soon,
Josh Gardner 1 Reputation point

2022-02-08T17:38:08.08+00:00

@Joyce Shen - MSFT Again, I appreciate your suggestions. All services are up, servers have been rebooted a few times just to be sure.

I went ahead and uninstalled Jan 2022 SU (KB5008631) and rebooted in my testbed, which did not remedy any of the errors
I went ahead and uninstalled Nov 2021 SU (KB5007409) and rebooted in my testbed, which did not remedy any of the errors

I suppose I could continue to roll back SU installations all the way back (due to the above testing, i'm on KB5007012 in my testbed), but I really dont know if there is a point. I haven't seen any errors in the event log that would indicate these DAG network errors are related to any Security Update with Exchange 2016 with CU22 installed. There is just.... nothing.

Is it somehow related to the CU?
Is it somehow related an older SU?
Is it somehow related to AD?
Is it somehow related to IIS?
Is it somehow related to .NET?
Joyce Shen - MSFT 16,541 Reputation points

2022-02-11T07:50:32.15+00:00

Hi,

Could you please try creating a new administrator account and make sure right permissions are granted to it then verify the result again?

And can you successfully create new DAG by running the New-DatabaseAvailabilityGroup command?
Josh Gardner 1 Reputation point

2022-02-16T19:09:40.347+00:00

@Joyce Shen - MSFT I'm pretty confident this error has nothing to do with permissions. I am in the Organization Management security group, and to my knowledge, I have zero issues running any other Exchange cmdlets beyond the DAG Network cmdlets

I was able to create a new IP-less DAG in my testbed (see picture 1). I did not add any servers to it HOWEVER, I was able to run at least ONE of the commands that I had not been able to before (see picture 2). What can explain this behavior please?
Josh Gardner 1 Reputation point

2022-02-16T20:34:44.263+00:00

As a follow up, I decided to back out of the original DAG in my testbed (TESTDAG). Since I was able to run the Set-DatabaseAvailabilityGroup -ManualDagNetworkConfiguration $True command on the new DAG in my testbed (TESTDAG2). I figured I'd remove my mailbox servers from the old and add them to the new. I also added a NIC to my testbed servers so that I can ensure that I can run DAG Network cmdlets. Sure enough, as soon as I added my first mailbox server into the new DAG and tried to run a DAG Network cmdlet it failed. Hopes have been dashed.
Joyce Shen - MSFT 16,541 Reputation points

2022-02-21T09:02:08.593+00:00

Hi @Josh Gardner

I would recommend you open a support ticket with Microsoft to solve this issue.

Global Customer service Phone number:
https://support.microsoft.com/en-us/topic/global-customer-service-phone-numbers-c0389ade-5640-e588-8b0e-28de8afeb3f2

If you have contract of paid support for on-premises Microsoft products and service, submit the ticket from this portal:
https://support.microsoft.com/en-us/gp/contactus81?audience=commercial
JBerg712 6 Reputation points

2022-02-22T20:57:33.817+00:00

Josh,

We have the exact same issue. We are migrating from Exchange 2013 to Exchange 2019 on the latest CU. For whatever reason, we cannot configure the DAG network because of the same error. It's very odd that the only error in the event logs is the cmdlet error, but states no reason as to why or what this is referring to exactly. I've turned up my event logging to EXPERT level and it's not recording anything when you attempt a network dag configuration change.

I'm working on it as well, so if I find anything out, i'll report back here.

JB
Josh Gardner 1 Reputation point

2022-02-23T17:44:53.71+00:00

@JBerg712 Thanks so much. My IT Leadership is trying to convert some stuff into eligible Microsoft Technical Support credits. I'm kind of stuck at the moment in that regard. I tested yet a 3rd Exchange environment that we have, and that one also gave me the same error. To sum up, all 3 Exchange environments that I manage cannot run Set-DatabaseAvailabilityGroupNetwork commands at all. Its very frustrating.

I will say, I am in the DoD so that "may" come into play because we harden things in a very particular way, but its all documented and I've used my testbed Exchange environment quite thoroughly to try and essentially create a "blank slate" Exchange enterprise environment (literally zero GPOs applied to the Exchange Servers, no HBSS/Firewalls, etc) and I still run into the error. At first I thought it may be a due to a hardening setting, but now I'm really thinking it may be a bug in the CU (not the SU, because I even uninstalled those in the testbed). I would GREATLY appreciate an update for any resolution if you are able to find one. Thanks again for posting.
Adnan Abdul Salam 96 Reputation points

2022-03-07T13:09:37.693+00:00

i am facing the same issue , not able to run the command or configure from GUI , i need to disable replication on MAPI network but below error comes either i do from GUI or Cmdlet

A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetwork

i have Jan SU installed ,

Firewall is disabled. no antivirus firewall etc.
Josh Gardner 1 Reputation point

2022-03-09T16:00:09.267+00:00

@Adnan Abdul Salam Thanks for posting, I'm hoping more and more people will comment. Maybe this will gain visibility and traction with the Microsoft engineers and they can dive into it. More people reporting an issue makes me think there is a bug somewhere.

Answer 1

Vikas Shah 6

@Josh Gardner

Did you find solutions? I am also facing a same issue while running Set-DatabaseAvailabilityGroupNetwork. i am trying to disable replication on MAPI nework configuration and getting same error message.

I do have same CU22 Jan SU installed in all Exchange Servers, so could it be the version issue? Did you tried running command in lower version CU?

Note: I can get the result while running Get-DatabaseAvailabilityGroupNetwork, but not working for Set command.

Josh Gardner 1 Reputation point

2022-02-04T13:18:50.387+00:00

@Vikas Shah Unfortunately not, though [sadly] I'm glad its not just me. I also have no issues with Get- commands, only the Set- commands. I was tempted to uninstall the Jan 2022 SU in my test bed to see if that would fix it, but... that seems drastic, and I dont know the impact. Hopefully it would just roll back to the Nov SU (but I really doubt it, because MS likes to do cumulative installs now).
Josh Gardner 1 Reputation point

2022-02-08T18:42:48.813+00:00

@Vikas Shah Just to let you know, I tried running the Set- commands after removing both the Jan 2022 SU and the Nov 2021 SU for Exchange 2016 CU 22. There was no change.

Answer 2

HaileSelassie 21

Same issue here, also with latest version Exchange 2019 CU11 March 2022 Security Updates. We have created an incident at Microsoft now. Will post any relevant updates

Josh Gardner 1 Reputation point

2022-03-09T16:02:39.507+00:00

@HaileSelassie Thanks for posting, I'm hoping more and more people will comment. Maybe this will gain visibility and traction with the Microsoft engineers and they can dive into it. More people reporting an issue makes me think there is a bug somewhere. Thank you very much for creating an incident with Microsoft, sadly we are still trying to convert things in order to get Microsoft support credit, so we are kind of stuck.

Answer 3

No news from MS case so far. I just realized that in the Exchange Team Blog "Released: January 2022 Exchange Server Security Updates" the issue has been added in the meantime. In the Blog Post "Released: March 2022 Exchange Server Security Updates" the issue has been mentioned as fixed:

"Modifying DAG network settings failing with error 0xe0434352 after January SUs are installed - fixed in the March SU"
Source: https://techcommunity.microsoft.com/t5/exchange-team-blog/released-march-2022-exchange-server-security-updates/ba-p/3247586

What I can confirm is, that after March SU, configuring "ManualDagNetworkConfiguration" attribute on the DAG is working for me. But, configuring DAGNetworks still throws the error. I also mentioned it here:

https://techcommunity.microsoft.com/t5/exchange-team-blog/released-march-2022-exchange-server-security-updates/bc-p/3254641/highlight/true#M32440

Answer 4

HaileSelassie 21

Seems like MS is aware of it:

Source: https://techcommunity.microsoft.com/t5/exchange-team-blog/released-march-2022-exchange-server-security-updates/bc-p/3254731/highlight/true#M32444

Josh Gardner 1 Reputation point

2022-03-11T14:06:16.62+00:00

@HaileSelassie Thanks for the update! I can only hope it is haha.

Answer 5

HaileSelassie 21

I can confirm that with Exchange 2019 CU12, configuring DAG Networks (Set-DatabaseAvailabilityGroupNetwork) is now possible again

Josh Gardner 1 Reputation point

2022-04-25T15:13:39.593+00:00

Thank you so much for the update. I'm going to install Exchange 2016 CU23 ASAP to confirm for us as well. Pity that MS did not post that they addressed the DAG cmdlets in the blog or release notes of the KB.
Josh Gardner 1 Reputation point

2022-04-26T19:49:16.093+00:00

I just installed Exchange 2016 CU23 in my test bed and I still cannot run Set-DatabaseAvailabilityGroupNetwork commands.....

@HaileSelassie Did you have to do anything besides install the CU?
Josh Gardner 1 Reputation point

2022-04-26T20:54:33.6+00:00

Nevermind, apparently MS didnt add it in. Of course.... I followed up with a comment in the blog.

https://techcommunity.microsoft.com/t5/exchange-team-blog/released-2022-h1-cumulative-updates-for-exchange-server/bc-p/3295680
HaileSelassie 21 Reputation points

2022-04-29T11:24:11.587+00:00

@Josh Gardner I verified again and I have to revise my statement, indeed I still receive the error...as also confirmed by MS not to be fixed in 2022 H1 CUs...don't know what I have verified the first time, sorry. So I was able to configure DAG and DAG Networks with Exchange 2019 CU11, after removing all Exchange Security Updates that have been applied after CU11.

Is anybody familiar with the error "A server-side administrative operation has failed. Operation failed with message: Error 0xe0434352 (Unknown error (0xe0434352)) from cli_SetDagNetworkConfig"

5 answers

Activity