SCCM Client Communication issue thru Zscaler VPN

Question

Hi,

we are having issue with SCCM Client those are off the company network and using Zscaler VPN to connect to corporate network. We have opened port for communication on firewall and Zscaler Admin server. We see that traffic are passing thru firewall and Zscaler but still client's are unable to assign site, MP etc.

Our Infra -

Single site with Cloud Management gateway and DP
Site boundaries are configured as per https://help.zscaler.com/zpa/supporting-microsoft-sccm
Client certificate is installed on client machine

ClientIDManagerStartup.log -

Machine: CGSURFXXXXX ClientIDManagerStartup 23/08/2021 14:39:22 13588 (0x3514)
OS Version: 10.0.19042.0 ClientIDManagerStartup 23/08/2021 14:39:22 13588 (0x3514)
SCCM Client Version: 5.00.9049.1010 ClientIDManagerStartup 23/08/2021 14:39:22 13588 (0x3514)
'RDV' Identity store does not support backup. ClientIDManagerStartup 23/08/2021 14:39:22 13588 (0x3514)
CCM Identity is in sync with Identity stores ClientIDManagerStartup 23/08/2021 14:39:22 13588 (0x3514)
Deleted Certificate ID from registry successfully ClientIDManagerStartup 23/08/2021 14:39:22 13588 (0x3514)
Client is set to use HTTPS when available. The current state is 224. ClientIDManagerStartup 23/08/2021 14:39:22 13588 (0x3514)
Generated a new Signing certificate ClientIDManagerStartup 23/08/2021 14:39:23 13588 (0x3514)
Generated a new Encryption certificate ClientIDManagerStartup 23/08/2021 14:39:23 13588 (0x3514)
[----- SHUTDOWN -----] ClientIDManagerStartup 23/08/2021 14:39:23 13588 (0x3514)
[----- STARTUP -----] ClientIDManagerStartup 23/08/2021 14:39:24 12540 (0x30FC)
Machine: CGSURFXXXXX ClientIDManagerStartup 23/08/2021 14:39:24 12540 (0x30FC)
OS Version: 10.0.19042.0 ClientIDManagerStartup 23/08/2021 14:39:24 12540 (0x30FC)
SCCM Client Version: 5.00.9049.1010 ClientIDManagerStartup 23/08/2021 14:39:24 12540 (0x30FC)
'RDV' Identity store does not support backup. ClientIDManagerStartup 23/08/2021 14:39:24 12540 (0x30FC)
CCM Identity is in sync with Identity stores ClientIDManagerStartup 23/08/2021 14:39:24 12540 (0x30FC)
Client is set to use HTTPS when available. The current state is 224. ClientIDManagerStartup 23/08/2021 14:39:24 12540 (0x30FC)
Registered AAD join event listener. ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
Registered for AAD on-boarding notifications. ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
[RegTask] - Executing registration task synchronously. ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
Read SMBIOS (encoded): 300030003600380035003300360039003200350035003300 ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
Evaluated SMBIOS (encoded): 300030003600380035003300360039003200350035003300 ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
No SMBIOS Changed ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
SMBIOS unchanged ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
SID unchanged ClientIDManagerStartup 23/08/2021 14:39:31 14956 (0x3A6C)
HWID unchanged ClientIDManagerStartup 23/08/2021 14:39:32 14956 (0x3A6C)
RegTask: Failed to refresh site code. Error: 0x8000ffff ClientIDManagerStartup 23/08/2021 14:39:42 14956 (0x3A6C)
Sleeping for 289 seconds before refreshing location services. ClientIDManagerStartup 23/08/2021 14:39:43 14956 (0x3A6C)

LocationService.Log -
Security settings update detected, restarting CcmExec. LocationServices 23/08/2021 14:39:23 13588 (0x3514)
LSRefreshSiteCode: Group Policy Updated the assigned site code <TTP>, which is different than the existing assigned site code <>. Will attempt re-assignment. LocationServices 23/08/2021 14:39:32 14956 (0x3A6C)
Sending Fallback Status Point message, STATEID='500'. LocationServices 23/08/2021 14:39:32 14956 (0x3A6C)
Processing GroupPolicy site assignment. LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Assigning to site 'TTP' LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
LSIsSiteCompatible : Verifying Site Compatibility for <TTP> LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Attempting to retrieve lookup MP(s) from AD LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Unexpected row count (0) retrieved from AD. LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
No lookup MP(s) from AD LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Attempting to retrieve lookup MP(s) from DNS LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Using default DNS suffix calor.co.uk LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Attempting to retrieve default management points from DNS LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Failed to retrieve DNS service record using _mssms_mp_ctp._tcp.ABC.co.uk lookup. DNS returned error 10057 LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
No lookup MP(s) from DNS LocationServices 23/08/2021 14:39:33 14956 (0x3A6C)
Failed to resolve 'SMS_SLP' from WINS LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
No lookup MP(s) from WINS LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Unable to find lookup MP(s) in Registry, AD, DNS and WINS LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Unexpected row count (0) retrieved from AD. LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
LSGetSiteVersionFromAD : Failed to retrieve version for the site 'TTP' (0x80004005) LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Attempting to retrieve lookup MP(s) from AD LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Unexpected row count (0) retrieved from AD. LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
No lookup MP(s) from AD LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Attempting to retrieve lookup MP(s) from DNS LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Using default DNS suffix ABC.co.uk LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Attempting to retrieve default management points from DNS LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Failed to retrieve DNS service record using _mssms_mp_ctp._tcp.ABC.co.uk lookup. DNS returned error 10057 LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
No lookup MP(s) from DNS LocationServices 23/08/2021 14:39:38 14956 (0x3A6C)
Failed to resolve 'SMS_SLP' from WINS LocationServices 23/08/2021 14:39:42 14956 (0x3A6C)
No lookup MP(s) from WINS LocationServices 23/08/2021 14:39:42 14956 (0x3A6C)
Unable to find lookup MP(s) in Registry, AD, DNS and WINS LocationServices 23/08/2021 14:39:42 14956 (0x3A6C)
LSIsSiteCompatible : Failed to get Site Version from all directories LocationServices 23/08/2021 14:39:42 14956 (0x3A6C)
Sending Fallback Status Point message, STATEID='608'. LocationServices 23/08/2021 14:39:42 14956 (0x3A6C)
Current AD site of machine is UK-Production LocationServices 23/08/2021 14:40:24 14472 (0x3888)

Answer 1

Hi,

We have solved the issue now by creating CNAME for (SMS_SLP.domain.com => SCCM server) and adding exception in Zscaler for _mssms_mp_SCCM Server FQDN_tcp.domain.com as client were doing name resolution for them. not sure why client was looking for SLP but these have been noticed in packet capturing log of Zscaler VPN client.

Answer 2

Hi @Rakesh Kumar ,

According to the information, it seems that these clients could not find the MPlist.

We could check if MP is published to DNS and AD on one client. Please refer to these following screenshot:

Besides, we could reinstall the client on one client, kindly specify SMSMP and SMSSITE on the command line.

---

If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 3

Hi @Amandayou-MSFT

yes all the entries as per screenshot shared by you are there in DNS and Adsiedit. I did multiple time installation of client but every time result is same.

below are the command lines used on multiple laptops.

Command line used -

ccmsetup.exe /mp:https://ABCCMG.CLOUDAPP.NET/CCM_Proxy_MutualAuth/XXXXX59403XXXXX CCMHOSTNAME=ABCCMG.CLOUDAPP.NET/CCM_Proxy_MutualAuth/XXXXX59403XXXXX SMSSITECODE=TTP SMSMP=https://SCCM01.ABC.COM AADTENANTID=XXXXXXX AADCLIENTAPPID=XXXXXXXXXXXXX AADRESOURCEURI=https://INABC-cg-configmgrservice

Token Based command line -
ccmsetup.exe /mp:https://ABCCMG.CLOUDAPP.NET/CCM_Proxy_MutualAuth/XXXXX59403XXXXX CCMHOSTNAME=ABCCMG.CLOUDAPP.NET/CCM_Proxy_MutualAuth/XXXXX59403XXXXX SMSSiteCode=TTP SMSMP=SCCM01.ABC.COM /regtoken:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxxx

standard command line -
ccmsetup.exe /mp:sccm01.abc.com smssitecode=TTP FSP=sccm01.abc.com

just for testing purpose i have changed the registry entry for one of internal client and tried to install one package but no luck.

changes made on one of internal sccm client -

HKLM/Software/Microsoft/CCM/Security/ClientAlwaysOnInternet to 1 and restarted the SMS Agent host service.

CCMexec.log -

Begin searching client certificates based on Certificate Issuers CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
Certificate Issuer 1 [CN=ABCCMG.cloudapp.net] CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
Skipping Certificate [Thumbprint 12E2A2B16B95C352044E7C1AFC967C8B77385731] issued to 'TSVDiSCCMSTS1.abc.com' as root is 'CN=ABC Root CA, O=ABC, OU= IT, L=Hoossss, S=Zd-india, C=IN' CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
Completed searching client certificates based on Certificate Issuers CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
Unable to find any Certificate based on Certificate Issuers CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
Raising event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:9F324D1F-3682-42C4-8089-EF957B2C1EF7";
DateTime = "20210824075117.943000+000";
HRESULT = "0x87d00215";
ProcessID = 11316;
ThreadID = 10708;
};
CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
[CCMHTTP] ERROR: URL=https://ABCCMG.CLOUDAPP.NET/CCM_Proxy_ServerAuth/xxxxxxxxx/ccm_system_tokenauth/request, Port=443, Options=1472, Code=0, Text=CCM_E_BAD_HTTP_STATUS_CODE CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
[CCMHTTP] ERROR INFO: StatusCode=403 StatusText=Forbidden CcmExec 24/08/2021 08:51:17 10708 (0x29D4)
Raising event:

CcmExec    24/08/2021 08:51:18    10708 (0x29D4)  

[CCMHTTP] ERROR: URL=https://ABCCMG.CLOUDAPP.NET/CCM_Proxy_ServerAuth/XXXXXXXX/ccm_system_tokenauth/request, Port=443, Options=1472, Code=0, Text=CCM_E_BAD_HTTP_STATUS_CODE CcmExec 24/08/2021 08:51:18 10708 (0x29D4)
[CCMHTTP] ERROR INFO: StatusCode=403 StatusText=Forbidden CcmExec 24/08/2021 08:51:18 10708 (0x29D4)
Raising event:
instance of CCM_CcmHttp_Status
{
ClientID = "GUID:9F324D1F-3682-42C4-8089-EF957B2C1EF7";
DateTime = "20210824075118.099000+000";
HostName = "ABC.CLOUDAPP.NET";
HRESULT = "0x87d0027e";
ProcessID = 11316;
StatusCode = 403;
ThreadID = 10708;
};
CcmExec 24/08/2021 08:51:18 10708 (0x29D4)
Successfully queued RefreshSecuritySettingsEvent event. CcmExec 24/08/2021 08:51:18 10708 (0x29D4)
Successfully queued event on HTTP/HTTPS failure for server 'ABCCMG.CLOUDAPP.NET'. CcmExec 24/08/2021 08:51:18 10708 (0x29D4)
Post to https://ABCCMG.CLOUDAPP.NET/CCM_Proxy_MutualAuth/XXXXXXX/ccm_system/ request failed with 0x87d00231. CcmExec 24/08/2021 08:51:18 10708 (0x29D4)
Exiting recently resumed state. CcmExec 24/08/2021 08:51:32 6480 (0x1950)
User SID 'S-1-5-21-1482476501-839522115-725345543-31035' unlock processing. CCMEXEC 24/08/2021 08:51:41 6480 (0x1950)
SystemTaskProcessor::QueueEvent(Unlock, 0) CCMEXEC 24/08/2021 08:51:41 6480 (0x1950)
BEGIN ExecuteSystemTasks('Unlock') CcmExec 24/08/2021 08:51:41 7120 (0x1BD0)
Invoking system task 'ComplRelayAgentUnlockTask' via ICcmSystemTask2 interface. CcmExec 24/08/2021 08:51:41 6480 (0x1950)
Invoking system task 'CertEnrollAgentUnlockTask' via ICcmSystemTask2 interface. CcmExec 24/08/2021 08:51:41 8848 (0x2290)
Invoking system task 'PolicyEvaluator_Unlock' via ICcmSystemTask2 interface. CcmExec 24/08/2021 08:51:41 10708 (0x29D4)
END ExecuteSystemTasks('Unlock') CcmExec 24/08/2021 08:51:41 7120 (0x1BD0)
SystemTaskProcessor::QueueEvent(PowerChanged, 0) CCMEXEC 24/08/2021 09:01:25 592 (0x0250)
BEGIN ExecuteSystemTasks('PowerChanged') CcmExec 24/08/2021 09:01:25 6480 (0x1950)
[Resource-Idle] User is away CCMEXEC 24/08/2021 09:01:25 592 (0x0250)
SystemTaskProcessor::QueueEvent(PowerChangedEx, 0) CCMEXEC 24/08/2021 09:01:25 592 (0x0250)
Invoking system task 'PowerStateManager_PowerChanged' via ICcmSystemTask2 interface. CcmExec 24/08/2021 09:01:25 8848 (0x2290)
Invoking system task 'PwrMgmtPowerChanged' via ICcmSystemTask2 interface. CcmExec 24/08/2021 09:01:25 8848 (0x2290)
END ExecuteSystemTasks('PowerChanged') CcmExec 24/08/2021 09:01:25 6480 (0x1950)
BEGIN ExecuteSystemTasks('PowerChangedEx') CcmExec 24/08/2021 09:01:25 10708 (0x29D4)
Invoking system task 'PwrMgmtPowerChangedEx' via ICcmSystemTask2 interface. CcmExec 24/08/2021 09:01:25 10136 (0x2798)
END ExecuteSystemTasks('PowerChangedEx') CcmExec 24/08/2021 09:01:25 10708 (0x29D4)
User SID 'S-1-5-21-1482476501-839522115-725345543-31035' lock processing. CCMEXEC 24/08/2021 09:01:25 10136 (0x2798)
SystemTaskProcessor::QueueEvent(Lock, 0) CCMEXEC 24/08/2021 09:01:25 10136 (0x2798)
BEGIN ExecuteSystemTasks('Lock') CcmExec 24/08/2021 09:01:25 10708 (0x29D4)
END ExecuteSystemTasks('Lock') CcmExec 24/08/2021 09:01:25 10708 (0x29D4)
SystemTaskProcessor::QueueEvent(PowerChanged, 0) CCMEXEC 24/08/2021 09:01:25 592 (0x0250)
BEGIN ExecuteSystemTasks('PowerChanged') CcmExec 24/08/2021 09:01:25 10136 (0x2798)