Troubleshooting on-premise environment deployment D365FFO

Artikel
04/17/2018

This document contains tips for troubleshooting on-premises Dynamics 365 for Finance and Operations environment deployment failures, based on my own experiences when troubleshooting this process for the first time.

Types of failures

The first type of failure I am seeing here is a simple redeploy of the environment. Originally I was trying to deploy a custom package, but it failed and I didn’t know why, so I deleted the environment and was redeploying with vanilla – no custom bits, just the base, and it still failed. In LCS after it runs for approx. 50 minutes I see the state change to Failed. There is no further log information in LCS itself, that information is within the respective machines in the on-premises environment.

Orchestrators

The Orchestrator machines trigger the deployment steps. In the base topology there are 3 orchestrators, these are clustered/load balanced – often the first one will pick up work, but don’t rely on that – it could be any of them which picks up tasks – and it could be more than 1 for a given deployment run – for example server 1 picks up some of the tasks and server 2 picks up some other tasks – always check event logs on all of them to avoid missing anything useful.

To make it easier to check them you can add a custom view of the event logs on each orchestrator machine, to give you all the necessary logs in one place, like this:
Create custom event log view Select events

I found in my case that server 2 was showing an error, as below, it’s basically saying it couldn’t find the AOS metadata service, and I notice the URL is wrong – I’d entered something wrong in the deployment settings in LCS:
Example error

AOS Machines

There are also useful logs on the AOS machines – the orchestrators are calling deployment scripts but for AX specific functions the AOSes are still running – for example database synchronize is run by an AOS. Again the AOSes are clustered so we need to check all of them as tasks could be executed by any of them. Similar to the orchestrators I create a custom event log view to show me all Dynamics related events in one place. This time I am selecting the Dynamics category, and I have unchecked “verbose” to reduce noise.

AOS event log

Here’s an example of a failure I had from a Retail deployment script which was trying to adjust a change tracking setting, for an issue such as this, once I know the error I can work around the problem by manually disabling change tracking on the problem table from SQL Server Management Studio and then starting the deployment again from LCS.

AOS example error

ADFS Machines

The ADFS servers will show authentication errors – typical causes of this kind of failure could be a “bad” setting entered in the deployment settings in LCS – for example I entered the DNS address for the ax instance incorrectly, then I see an ADFS error after deployment when trying to log into AX:

ADFS error example

If you see an error as above, you can understand more about it my reviewing the Application group setup in “ADFS Management” on the ADFS machine, open it from server manager:

ADFS

Under application groups you’ll see one for D365, double click it to see the details

ADFS setup

If you’re familiar with the cloud version of D365, then you’ll probably know that AAD requires application URLs to be configured against it to allow you to log in – in cloud the deployment process from LCS is doing this automatically, and you can see it if you review your AAD setup via the Azure portal. In the on-prem version, this ADFS management tool shows you the same kind of setup, also in on-prem the deployment process is creating these entries automatically for you. Click on one of the native applications listed and then the edit button you can see what’s been set up:

ADFS application group setup

The authentication error I mentioned previously:
MSIS9224: Received invalid OAuth authorization request. The received 'redirect_uri' parameter is not a valid registered redirect URI for the client identifier: 'f06b0738-aa7a-4a50-a406-5c1e486c49be'. Received redirect_uri: 'https://dax7sqlaodc1.saonprem.com/namespaces/AXSF/'.

We can now see from the configuration above that for client 'f06b0738-aa7a-4a50-a406-5c1e486c49be' the request URL isn’t configured. If we believed that the URL is correct, then we could add it here and ADFS would then allow the request to go through successfully. IN my case the URL was the mistake, so I didn’t change ADFS settings, I corrected the URL in LCS and started the deployment again.

Package deployment failures

When reconfiguring an environment, and including a custom package, if the deployment fails, check the orchestrator machine event logs, as described above – use a custom event log view to check all the logs on a machine at once.

I have had a situation where I’m getting failures related to package dependencies where my package does not have the failing dependency, I will explain:
Error is:

Package [dynamicsax-demodatasuite.7.0.4679.35176.nupkg has missing dependencies: [dynamicsax-applicationfoundationformadaptor;dynamicsax-applicationplatformformadaptor;dynamicsax-applicationsuiteformadaptor]]

My package does not contain demodatasuite, so the error is a mystery. Turns out that because my package has the same filename as a previously deployed package, the system is not downloading my package and just attempting to deploy an old package with the same name. Packages can be found in the file share location, as below:
\\DAX7SQLAOFILE1\SQLFileShare\assets

The first part, \\DAX7SQLAOFILE1\SQLFileShare, is my file share (so will differ in different environments – it’s a setting given when the environment was created), the assets folder is constant.

In here I see that my current package “a.zip” (renamed to a short name to work around an issue with deployment failure due to path too long), is from several weeks ago and is much larger than the package I expect. To get past this I rename my package to b.zip and attempt deployment again. Note that after PU12 for on-premises this issue no longer occurs.

Package deployment process

During the package deployment process, the combined packages folders will be created in this folder:

\\DAX7SQLAOFILE1\SQLFileShare\wp\Prod\StandaloneSetup-109956\tmp\Packages

Error when environment left in Configuration mode

When running a reployment, the error below can occur if the environment has been left in Configuration mode (for changing config keys), turn off configuration mode, restart the AOSes and then re-run the deployment.

MachineName SQLAOSF1ORCH2 EnvironmentId c91bafd5-ac0b-43dd-bd5f-1dce190d9d49 SetupModuleName FinancialReporting Component Microsoft.Dynamics.Performance.Deployment.Commands.AX.AddAXDatabaseChangeTracking Message An unexpected error occurred while querying the Metadata service. Check that all credentials are correct. See the deployment log for details. Detail Microsoft.Dynamics.Performance.Deployment.Common.DeploymentException: An unexpected error occurred while querying the Metadata service. Check that all credentials are correct. See the deployment log for details. ---> System.ServiceModel.FaultException: Internal Server Error Server stack trace: at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at

Error when FRServiceUser is missing

This error can also happen when the FRServiceUser is missing in USERINFO – the AOS metadata service is trying to create an AX session as this user.
This user is normally created by the DB synch process. If the user is incorrect in USERINFO then deleting that User and re-running DB synch should recreate the user – you can set USERINFO.ISMICROSOFTACCOUNT to 0 in SSMS, and then re-run db synch to create the user. DB synch can be triggered in PU12+ by clearing the SF.SYNCLOG table and then killing AXService.exe – when it automatically starts back up it will run a db synch. The you should see the FRServiceUser created back in USERINFO.

Delen via