Coding from the field

Resetting WMI repository - dos and dont's

Michael Frommhold MSFT — Wed, 10 Oct 2018 11:32:14 GMT

Today's topic:

Resetting WMI Repository

dos and dont's

Some people like Windows Management Instrumentation (WMI) because of its power - some just hate it. The ones who hate it often claim that the WMI repository is not really reliable and often gets corrupt / unusable. Even though this may happen - it's not a usual scenario. Anyhow - if you face a corrupt WMI repository - you need to repair it. The basic steps would be

set WinMgmt service state to Disabled
stop WinMgmt service
walk through all dlls in %windir%\system32\wbem and re-register those
call /regserver method on WMI private server executable (wmiprvse.exe)
call /resetrepository method on WinMgmt service executable to reset the WMI repository to the initial state and restore MOF files that contain the #pragma autorecover statement
set WinMgmt service state to Auto
start WinMgmt service
walk through all relevant Windows Management Object Files (.mof) in %windir%\system32\wbem and all Windows Management Language Files (.mfl) in language subfolders of %windir%\system32\wbem and re-register them by mofcomp.exe calls.
restart machine

There are various articles out there that describe how to do it - some even provide automation code. Unfortunately most of these disregard the fact that the purpose of some MOF files is to uninstall the WMI namespace / class defined in the MOF - they just run through all MOF files and register those. Saying - if you mofcomp all MOF files found in the wbem folder - you may end up with an even more unusable WMI repository - just because you uninstalled necessary WMI data ... How you detect MOFs / MFLs that are intended to uninstall WMI namespaces / classes: Some MOFs / MFLs are very kind to the admins and have 'uninstall' or 'remove' as part of their name - but this does not detect all classes that uninstall WMI data. To reliably identify the MOFs / MFLs to be registered by mofcomp -> do the following:

skip MOFs / MFLs that have 'autorecover' as part of their name
skip MOFs / MFLs that have 'uninstall' or 'remove' as part of their name
skip MOFs / MFLs that contain the #pragma autorecover statement (already handled by winmgmt /resetrepository)
skip MOFs / MFLs that do not contain the #pragma autorecover statement but either #pragma deleteinstance or #pragma deleteclass statement(s)

To bring this all together in some PoSh code: [code lang="powershell"] function DisableService([System.ServiceProcess.ServiceController]$svc) { Set-Service -Name $svc.Name -StartupType Disabled } function EnableServiceAuto([System.ServiceProcess.ServiceController]$svc) { Set-Service -Name $svc.Name -StartupType Automatic } function StopService([System.ServiceProcess.ServiceController]$svc) { [string]$dep = ([string]::Empty) foreach ($depsvc in $svc.DependentServices) { $dep += $depsvc.DisplayName + ", " } Write-Host "Stopping $($svc.DisplayName) and its dependent services ($dep)" $svc.Stop() $svc.WaitForStatus([System.ServiceProcess.ServiceControllerStatus]::Stopped) Write-Host "Stopped $($svc.DisplayName)" } function StartService([System.ServiceProcess.ServiceController]$svc, [bool]$handleDependentServices) { if ($handleDependentServices) { Write-Host "Starting $($svc.DisplayName) and its dependent services" } else { Write-Host "Starting $($svc.DisplayName)" } if (!$svc.Status -ne [System.ServiceProcess.ServiceControllerStatus]::Running) { try { $svc.Start() $svc.WaitForStatus([System.ServiceProcess.ServiceControllerStatus]::Running) } catch { } } Write-Host "Started $($svc.DisplayName)" if ($handleDependentServices) { [System.ServiceProcess.ServiceController]$depsvc = $null; foreach ($depsvc in $svc.DependentServices) { if ($depsvc.StartType -eq [System.ServiceProcess.ServiceStartMode]::Automatic) { StartService $depsvc $handleDependentServices } } } } function RegSvr32([string]$path) { Write-Host "Registering $path" regsvr32.exe $path /s } function RegisterMof([System.IO.FileSystemInfo]$item) { [bool]$register = $true Write-Host "Inspecting: $($item.FullName)" if ($item.Name.ToLowerInvariant().Contains('uninstall')) { $register = $false Write-Host "Skipping - uninstall file: $($item.FullName)" } elseif ($item.Name.ToLowerInvariant().Contains('remove')) { $register = $false Write-Host "Skipping - remove file: $($item.FullName)" } else { $txt = Get-Content $item.FullName if ($txt.Contains('#pragma autorecover')) { $register = $false Write-Host "Skipping - autorecover: $($item.FullName)" } elseif ($txt.Contains('#pragma deleteinstance')) { $register = $false Write-Host "Skipping - deleteinstance: $($item.FullName)" } elseif ($txt.Contains('#pragma deleteclass')) { $register = $false Write-Host "Skipping - deleteclass: $($item.FullName)" } } if ($register) { Write-Host "Registering $($item.FullName)" mofcomp $item.FullName } } function HandleFSO([System.IO.FileSystemInfo]$item, [string]$targetExt) { if ($item.Extension -ne [string]::Empty) { if ($targetExt -eq 'dll') { if ($item.Extension.ToLowerInvariant() -eq '.dll') { RegSvr32 $item.FullName } } elseif ($targetExt -eq 'mof') { if (($item.Extension.ToLowerInvariant() -eq '.mof') -or ($item.Extension.ToLowerInvariant() -eq '.mfl')) { RegisterMof $item } } } } # get Winmgmt service [System.ServiceProcess.ServiceController]$wmisvc = Get-Service 'winmgmt' # disable winmgmt service DisableService $wmisvc # stop winmgmt service StopService $wmisvc # get wbem folder [string]$wbempath = [Environment]::ExpandEnvironmentVariables("%windir%\system32\wbem") [System.IO.FileSystemInfo[]]$itemlist = Get-ChildItem $wbempath -Recurse | Where-Object { $_.FullName.Contains('AutoRecover') -ne $true} [System.IO.FileSystemInfo]$item = $null # walk dlls foreach ($item in $itemlist) { HandleFSO $item 'dll' } # call /regserver method on WMI private server executable wmiprvse /regserver # call /resetrepository method on WinMgmt service executable winmgmt /resetrepository # enable winmgmt service EnableServiceAuto $wmisvc # start winmgmt service StartService $wmisvc $true # walk MOF / MFLs foreach ($item in $itemlist) { HandleFSO $item 'mof' } [/code] Kudos to my dear colleague Constantin - one of the WMI haters :- ) Have fun resetting... 4/6/2019: Updated PoSh code to only start dependent services if start mode of those is set to 'Automatic' Michael Have keyboard. Will travel.

Netlogon Debug helper

Michael Frommhold MSFT — Thu, 03 May 2018 10:45:00 GMT

Today's topic:

Netlogon Debug helper

Sometimes it can be helpfull to get inforamtion about actions performed by the Netlogon service on a machine. To gather such information you need to enable Netlogon Debug Logging. Just to make things easier - you will find download links at the end of the article for a little helper program that:

helps building the debug flag bit mask corresponding to your needs
explains the bits int the debug flag bit mask
sets the desired value in the registry
restats Netlogon service

Some basics - to configrure Netlogn Debug Logging we need to set a DWORD value in the registry. Path to parent registry key:

HKLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters

Registry value:

DBFlag = desired debug flag bit mask

The corresponding log entries can be found in:

%windir%\debug\netlogon.log

To control log file size (default = 20 MB) we may use the follwing DWORD value in the same registry key:

MaximumLogFileSize = maximum log file size in bytes

There are many articles out there talking about this topic - but unfortunately they all miss detailed information about the valid debug flags that can be configured. Most of the articles are suggesting to set value 0x2080ffff - which results in very verbose debug reporting -> white noise. Apart form complexity of the analysis of the huge amount of the gathered data - your log file may be overwritten very quickly -> thus you may miss log entries of interest. Therefore, I suggest to only set those bits for debug flags that will let netlogon log only the data you are interested in. List of the debug flag bits: [code lang="c-sharp"] [Flags] public enum DEBUG_FLAGS : uint { NL_INIT = 0x00000001, // Netlogon initialization NL_MISC = 0x00000002, // Misc debug (like DCLocator) NL_LOGON = 0x00000004, // Logon processing NL_SYNC = 0x00000008, // Synchronization and replication NL_MAILSLOT = 0x00000010, // Mailslot messages NL_SITE = 0x00000020, // Sites NL_MSA = 0x00000040, // Managed Service Account Scavenger processing NL_1 = 0x00000080, // Unknown NL_CRITICAL = 0x00000100, // Only real important errors NL_SESSION_SETUP = 0x00000200, // Trusted Domain maintenance NL_DOMAIN = 0x00000400, // Hosted Domain maintenance NL_2 = 0x00000800, // UnKnown NL_SERVER_SESS = 0x00001000, // Server session maintenance NL_CHANGELOG = 0x00002000, // Change Log references NL_DNS = 0x00004000, // DNS name registration // Verbose bits NL_VERBOSE = 0x00008000, // Enable verbose logging NL_WORKER = 0x00010000, // Debug worker thread NL_DNS_MORE = 0x00020000, // Verbose DNS name registration NL_PULSE_MORE = 0x00040000, // Verbose pulse processing NL_SESSION_MORE = 0x00080000, // Verbose session management NL_REPL_TIME = 0x00100000, // replication timing output NL_REPL_OBJ_TIME = 0x00200000, // replication objects get/set timing output NL_ENCRYPT = 0x00400000, // debug encrypt and decrypt across net NL_SYNC_MORE = 0x00800000, // additional replication dbgprint NL_PACK_VERBOSE = 0x01000000, // Verbose Pack/Unpack NL_MAILSLOT_TEXT = 0x02000000, // Verbose Mailslot messages NL_CHALLENGE_RES = 0x04000000, // challenge response debug NL_SITE_MORE = 0x08000000, // Verbose sites // Control bits. NL_INHIBIT_CANCEL = 0x10000000, // Don't cancel API calls NL_TIMESTAMP = 0x20000000, // TimeStamp each output line NL_ONECHANGE_REPL = 0x40000000, // Only replicate one change per call NL_BREAKPOINT = 0x80000000 // Enter debugger on startup } [/code] Example - you are interested in logon processing -> set debug flag value NL_TIMESTAMP | NL_LOGON (0x20000004). Keep in mind - only if you set bit NL_TIMESTAMP (0x20000000) you will have the time stamp for each log entry in the netlogon.log. FYI - white noise DBFlag value 0x2080fff translates to:

NL_INIT
NL_MISC
NL_LOGON
NL_SYNC
NL_MAILSLOT
NL_SITE
NL_MSA
NL_1
NL_CRITICAL
NL_SESSION_SETUP
NL_DOMAIN
NL_2
NL_SERVER_SESS
NL_CHANGELOG
NL_DNS
NL_VERBOSE
NL_SYNC_MORE
NL_TIMESTAMP

Netlogon Debug Helper Tool downloads:

Compiled sample code (compiled targeting .Net frameworks 3.5 and 4.5) NetlogonDebug_Compiled
Download Visual Studio sample code project CodingFromTheField.NetlogonDebug

All the best and have fun debugging. Michael PFE | Have keyboard. Will travel

Automated Offline Domain Join using PowerShell and JEA

Raimund Andrée — Fri, 27 Oct 2017 08:25:21 GMT

Abstract

An offline domain join requires network access to a writable Domain Controller, which might not be desired in a DMZ scenario. Also, permissions to create computer accounts are required and permissions to enable password replication to the Read-Only Domain Controller. This document describes the process of delegating and fully automating this process by means of PowerShell Restricted Endpoints. The setup of the endpoints is also part of this document.

Expected Result

After completing the steps outlined in the document, dedicated user accounts with no administrative permission in Active Directory can do an offline domain join without the need to contact a writable Domain Controller. The offline domain join can be done completely from the machine to join and no further steps are required on the Read-Only nor a writable Domain Controller. This process offers a very high level of comfort while giving the least privileges required to the users responsible for joining machines into the Active Directory.

Audience

Technical personnel responsible to managing Active Directory and security officers.

Requirements

An Active Directory running at least Windows Server 2008 R2. Clients and servers in the DMZ can only communicate to Read-Only Domain Controllers. Writable Domain Controllers cannot be reached from the DMZ. Windows Management Framework 5.1 must be installed on the servers that will host the PowerShell Restricted Endpoints. A user account with no additional privileges but having the right to create computer accounts and add someone to the group "Allowed RODC Password Replication Group".

Technical documentation

PowerShell Restricted Endpoints - background

A custom PowerShell Restricted Endpoints works like the default endpoints that already exist on every computer with at least WMF2. When creating new endpoints, you can assign permissions so that only selected users can access this new endpoint. Furthermore, these endpoints can be configured to run with predefined credentials. This can be either domain credentials or a local virtual account. In this scenario, both types are used. To control what the users working with a custom endpoint are allowed to do, PowerShell works in a white-list mode. Only commands and functions that are explicitly allowed can be used within the endpoint. In this scenario only a single function is allowed. Further documentation can be found here:

Documentation how to further extend this concept with JEA:

Just Enough Administration

PowerShell Restricted Endpoints in this specific scenario

The process in this scenario requires two endpoints. One proxy endpoint in the DMZ that the clients and servers to join can reach. The other endpoint is in the internal network and can only be reached from the proxy endpoint in the DMZ. The proxy endpoint in the DMZ just forwards the request to the one in the internal network. The workflow is:

The client or server that should join the domain connects to the proxy endpoint in the DMZ.
The proxy endpoint just forwards the join request to the endpoint in the internal network.
The endpoint in the internal network does all the work to get the machine joined to the domain.
- It checks if the site name is correct and the OU exists.
- It creates the computer account using DJOIN.exe in the given OU. DJOIN.EXE creates a blob that must be stored on the machine to join. This blog is returned via the proxy endpoint to the machine requesting the domain join.
- The machine is added to the group "Allowed RODC Password Replication Group".
- If requested (PrepopulatePassword), the password can be pre-populated. This requires some special rights that are not part of this paper.

Security Aspects

None of the accounts used has administrative access to the Active Directory. The only access delegated to the user account running the internal endpoint is:

Creating computer accounts in the dedicated OUs
Adding a computer or user to the group "Allowed RODC Password Replication Group".

Even if the machines hosting the endpoints are compromised, no critical credentials are effected.

The Deployment

Note: All the scripts referenced are in the appendix at the bottom.

Two machines are required to deploy the solution, one in the DMZ and one in the internal network. The endpoints should not be deployed to domain controllers for security reasons. A service account is required for the endpoint running in the internal network. The required steps to deploy the solution are:

Service Account
The endpoint in the internal network runs with domain user credentials. The service user must have the right to create computer accounts in the particular OUs and to alter the member of the group "Allowed RODC Password Replication Group". The following commands create the user account and add the required permissions to the computers container and a test OU. It also grants the service account permissions to change the membership of the "Allowed RODC Password Replication" group. [code lang="PowerShell" gutter="false"] New-ADUser -Name JoinUser -AccountPassword ('Password1' | ConvertTo-SecureString -AsPlainText -Force) -Enabled $true dsacls "CN=Allowed RODC Password Replication Group,CN=Users,DC=contoso,DC=com" /G "contoso\OfflineDomainJoin:WP;member;" dsacls "OU=Test,DC=contoso,DC=com" /G "contoso\OfflineDomainJoin:GRGE;computer" dsacls "CN=Computers,DC=contoso,DC=com" /G "contoso\OfflineDomainJoin:GRGE;computer" [/code]
Create the endpoint in the internal network
Run the script in section RestrictedEndpoint LAN.ps1 to deploy the endpoint. Please change the highlighted section according to your environment. [code lang="PowerShell" gutter="false"] Register-SupportPSSessionConfiguration -RunAsUser contoso\OfflineDomainJoin -RunAsUserPassword Password1 -AllowedPrincipals contoso\zMgmtRodcs$ -Force [/code]
Note: contoso\zMgmtRodcs$ must be replaced with the computer account of the machine in the DMZ where the proxy endpoint is running on. Please make sure that there is a $ at the end of the argument.
Create the proxy endpoint in the internal network
The script in the section RestrictedEndpoint DMZ.ps1 creates the proxy endpoint in the DMZ. This endpoint needs to know where the internal endpoint can be found. Change this line in the parameter block accordingly. [code lang="PowerShell" gutter="false"] [string]$Server = 'zMgmtLan.contoso.com' [/code] ,The other line that must be changes it the function call at the very bottom. Please replace the highlighted part with the users or groups that should be able to join computers to the domain. If you have more than one principal, provide a comma-separated list. [code lang="PowerShell" gutter="false"] Register-SupportPSSessionConfiguration -UseVirtualAccount -AllowedPrincipals contoso\JoinUser -Force [/code] After having made the changes, run the script on the machine you want to have a proxy endpoint on.
Join a Machine Using the Automated Offline Domain Join
On the machine you want to offline-join to the domain, get the credentials for a user that is allowed to connect to the proxy endpoint. [code lang="PowerShell" gutter="false"] $cred = Get-Credential -Credential contoso\JoinUser [/code] Call the script in section OfflineDomainJoinRequest.ps1. The highlighted parts should be changed according to your environment. The parameter “Server” takes the name of the machine hosting the proxy endpoint. The given site name should match the Active Directory site the machine is part of. [code lang="PowerShell" gutter="false"] C:\OfflineDomainJoinRequest.ps1 -Server zMgmtRodcs.contoso.com -SiteName DMZ -OrganizationalUnit 'OU=Test,DC=contoso,DC=com' -DoNotRestart -Credential $cred [/code]
Note: The service account the internal endpoint is working with must have permissions on the given OU to create computer objects.

Appendix

RestrictedEndpoint LAN.ps1
[code lang="PowerShell"] function New-ADOfflineDomainJoin { param( [Parameter(Mandatory)] [string]$ComputerName, [string]$SiteName, [string]$OrganizationalUnit, [switch]$PrepopulatePassword ) $domain = Get-ADDomain -Current LocalComputer Write-Host "Current domain is '$($domain.DNSRoot)'" $rodcs = $domain.ReadOnlyReplicaDirectoryServers Write-Host "$($rodcs.Count) Read-Only Domain Controllers found: $($rodcs -join ', ')" $writableDC = (Get-ADDomainController -Writable -Discover).HostName[0] Write-Host "Writable Domain Controller is '$writableDC'" if ($SiteName) { try { Get-ADReplicationSite -Identity $SiteName | Out-Null } catch { Write-Error "The Active Directory site '$SiteName' could not be found" return } } if ($OrganizationalUnit) { try { Get-ADOrganizationalUnit -Identity $OrganizationalUnit | Out-Null } catch { Write-Error "The Active Directory OU '$OrganizationalUnit' could not be found" return } } Write-Host $tempFile = [System.IO.Path]::GetTempFileName() Remove-Item -Path $tempFile Write-Host "Calling DJOIN.EXE..." -NoNewline $cmd = 'djoin.exe /provision /domain "{0}" /MACHINE {1} /SAVEFILE {2} /DCName {3}' -f $domain.DNSRoot, $ComputerName, $tempFile, $writableDC if ($OrganizationalUnit) { $cmd += " /MACHINEOU $OrganizationalUnit" } if ($SiteName) { $cmd += " /PSITE $SiteName" } Write-Host 'Running the following djoin.exe command:' Write-Host $cmd $djoinResult = &([scriptblock]::Create($cmd)) if ($djoinResult -like '*Computer provisioning completed successfully*') { Write-Host 'successfull' } else { Write-Host "there was an error: $($djoinResult[-2])" return } Write-Host $computer = Get-ADComputer -Identity $ComputerName -Server $writableDC Write-Host "Adding computer account '$ComputerName' to group 'Allowed RODC Password Replication Group'" Add-ADGroupMember -Members $computer -Identity 'Allowed RODC Password Replication Group' -Server $writableDC Write-Host if ($PrepopulatePassword) { foreach ($rodc in $rodcs) { Write-Host "Prepopulating password for account '$($($computer.DistinguishedName))' to RODC '$rodc' from writable DC '$writableDC'..." -NoNewline $repadminResult = repadmin.exe /rodcpwdrepl $rodc $writableDC ""$($computer.DistinguishedName)"" if ($repadminResult -like '*Successfully replicated secrets*') { Write-Host 'successfull' } else { Write-Host 'error' Write-Error ($repadminResult -join '. ') return } } Write-Host } Get-Content -Path $tempFile Remove-Item -Path $tempFile } function Register-SupportPSSessionConfiguration { param( [Parameter(Mandatory, ParameterSetName = 'UserAccount')] [string]$RunAsUser, [Parameter(Mandatory, ParameterSetName = 'UserAccount')] [string]$RunAsUserPassword, [Parameter(Mandatory, ParameterSetName = 'VirtualAccount')] [switch]$UseVirtualAccount, [string[]]$AllowedPrincipals, [switch]$Force ) $modulesToImport = 'ActiveDirectory' $path = [System.IO.Path]::GetTempFileName() Remove-Item -Path $path $path = [System.IO.Path]::ChangeExtension($path, '.pssc') $endpointName = 'OfflineDomainJoin' if ($Force -and (Get-PSSessionConfiguration -Name $endpointName -ErrorAction SilentlyContinue)) { Get-PSSessionConfiguration -Name $endpointName | Unregister-PSSessionConfiguration } $param = @{} $param.Add('Path', $path) $param.Add('ModulesToImport', $modulesToImport) $param.Add('SessionType', 'Default') $param.Add('LanguageMode', 'FullLanguage') $param.Add('VisibleProviders', 'FileSystem') $param.Add('ExecutionPolicy', 'Unrestricted') $param.Add('Full', $true) if ($UseVirtualAccount) { $param.Add('RunAsVirtualAccount', $true) } $param.Add('FunctionDefinitions', @{ Name = 'New-ADOfflineDomainJoin' ScriptBlock = (Get-Command -Name New-ADOfflineDomainJoin).ScriptBlock } ) New-PSSessionConfigurationFile @param if ($RunAsUser) { $cred = New-Object pscredential($RunAsUser, ($RunAsUserPassword | ConvertTo-SecureString -AsPlainText -Force)) } $param = @{ Name = $endpointName Path = $path Force = $Force } if ($RunAsUser) { $param.Add('RunAsCredential', $cred) } try { Register-PSSessionConfiguration @param -ErrorAction Stop } catch { Write-Error -Exception $_.Exception return } finally { Remove-Item -Path $path } $pssc = Get-PSSessionConfiguration -Name $endpointName $psscSd = New-Object System.Security.AccessControl.CommonSecurityDescriptor($false, $false, $pssc.SecurityDescriptorSddl) foreach ($allowedPrincipal in $AllowedPrincipals) { $account = New-Object System.Security.Principal.NTAccount($allowedPrincipal) $accessType = "Allow" $accessMask = 268435456 $inheritanceFlags = "None" $propagationFlags = "None" $psscSd.DiscretionaryAcl.AddAccess($accessType,$account.Translate([System.Security.Principal.SecurityIdentifier]),$accessMask,$inheritanceFlags,$propagationFlags) } Set-PSSessionConfiguration -Name $endpointName -SecurityDescriptorSddl $psscSd.GetSddlForm("All") -Force } Register-SupportPSSessionConfiguration -RunAsUser contoso\OfflineDomainJoin -RunAsUserPassword Password1 -AllowedPrincipals contoso\zMgmtRodcs$ -Force [/code]
RestrictedEndpoint DMZ.ps1
[code lang="PowerShell"] function Request-ADOfflineDomainJoin { param( [Parameter(Mandatory)] [string]$ComputerName, [string]$SiteName, [string]$OrganizationalUnit, [string]$Server = 'zMgmtLan.contoso.com', [switch]$PrepopulatePassword ) $s = New-PSSession -ComputerName $Server -ConfigurationName OfflineDomainJoin $blob = Invoke-Command -Session $s -ScriptBlock { $param = @{ ComputerName = $using:ComputerName } if ($using:SiteName) { $param.Add('SiteName', $using:SiteName) } if ($using:OrganizationalUnit) { $param.Add('OrganizationalUnit', $using:OrganizationalUnit) } if ($using:PrepopulatePassword) { $param.Add('PrepopulatePassword', $true) } New-ADOfflineDomainJoin @param } $blob } function Register-SupportPSSessionConfiguration { param( [Parameter(Mandatory, ParameterSetName = 'UserAccount')] [string]$RunAsUser, [Parameter(Mandatory, ParameterSetName = 'UserAccount')] [string]$RunAsUserPassword, [Parameter(Mandatory, ParameterSetName = 'VirtualAccount')] [switch]$UseVirtualAccount, [string[]]$AllowedPrincipals, [switch]$Force ) $modulesToImport = 'ActiveDirectory' $path = [System.IO.Path]::GetTempFileName() Remove-Item -Path $path $path = [System.IO.Path]::ChangeExtension($path, '.pssc') $endpointName = 'OfflineDomainJoinProxy' if ($Force -and (Get-PSSessionConfiguration -Name $endpointName -ErrorAction SilentlyContinue)) { Get-PSSessionConfiguration -Name $endpointName | Unregister-PSSessionConfiguration } $param = @{} $param.Add('Path', $path) $param.Add('ModulesToImport', $modulesToImport) $param.Add('SessionType', 'Default') $param.Add('LanguageMode', 'FullLanguage') $param.Add('VisibleProviders', 'FileSystem') $param.Add('ExecutionPolicy', 'Unrestricted') $param.Add('Full', $true) if ($UseVirtualAccount) { $param.Add('RunAsVirtualAccount', $true) } $param.Add('FunctionDefinitions', @{ Name = 'Request-ADOfflineDomainJoin' ScriptBlock = (Get-Command -Name Request-ADOfflineDomainJoin).ScriptBlock } ) New-PSSessionConfigurationFile @param if ($RunAsUser) { $cred = New-Object pscredential($RunAsUser, ($RunAsUserPassword | ConvertTo-SecureString -AsPlainText -Force)) } $param = @{ Name = $endpointName Path = $path Force = $Force } if ($RunAsUser) { $param.Add('RunAsCredential', $cred) } try { Register-PSSessionConfiguration @param -ErrorAction Stop } catch { Write-Error -Exception $_.Exception return } finally { Remove-Item -Path $path } $pssc = Get-PSSessionConfiguration -Name $endpointName $psscSd = New-Object System.Security.AccessControl.CommonSecurityDescriptor($false, $false, $pssc.SecurityDescriptorSddl) foreach ($allowedPrincipal in $AllowedPrincipals) { $account = New-Object System.Security.Principal.NTAccount($allowedPrincipal) $accessType = "Allow" $accessMask = 268435456 $inheritanceFlags = "None" $propagationFlags = "None" $psscSd.DiscretionaryAcl.AddAccess($accessType,$account.Translate([System.Security.Principal.SecurityIdentifier]),$accessMask,$inheritanceFlags,$propagationFlags) } Set-PSSessionConfiguration -Name $endpointName -SecurityDescriptorSddl $psscSd.GetSddlForm("All") -Force } Register-SupportPSSessionConfiguration -UseVirtualAccount -AllowedPrincipals contoso\JoinUser -Force [/code]
OfflineDomainJoinRequest.ps1
[code lang="PowerShell"] param( [Parameter(Mandatory)] [string]$Server, [Parameter(Mandatory)] [pscredential]$Credential, [string]$SiteName, [string]$OrganizationalUnit, [switch]$PrepopulatePassword, [switch]$DoNotRestart ) Set-Item -Path WSMan:\localhost\Client\TrustedHosts -Value $Server -Force $computerName = $env:COMPUTERNAME $s = New-PSSession -ComputerName $Server -ConfigurationName OfflineDomainJoinProxy -Credential $Credential -ErrorAction Stop $blob = Invoke-Command -Session $s -ScriptBlock { $param = @{ ComputerName = $using:ComputerName } if ($using:SiteName) { $param.Add('SiteName', $using:SiteName) } if ($using:OrganizationalUnit) { $param.Add('OrganizationalUnit', $using:OrganizationalUnit) } if ($using:PrepopulatePassword) { $param.Add('PrepopulatePassword', $true) } Request-ADOfflineDomainJoin @param } if (-not $blob) { Write-Error 'Failed to retreive blob for offline domain join' return } $tempFile = [System.IO.Path]::GetTempFileName() $blob | Set-Content -Path $tempFile -Encoding Unicode cmd /c DJOIN /REQUESTODJ /LOADFILE $tempFile /WINDOWSPATH %windir% /LOCALOS Remove-Item -Path $tempFile if (-not $DoNotRestart) { Restart-Computer -Force } [/code]

Deploy an ADLDS lab with replication and sync in less than an hour.

Raimund Andrée — Mon, 14 Aug 2017 22:03:51 GMT

Abstract

This article helps you creating an ADLDS lab in less than an hour. If you follow the article as it is, you will have an AD forest with an Enterprise Certificate Authority and 4 servers not joined to the domain. These 4 servers will hold an ADLDS instances that syncs some data from the AD forest. LDAPS is enabled for the ADLDS instance. Of course, the lab can be made simpler and you can skip parts of the deployment process.

Audience

If you are an ADLDS expert and want to save time, this deployment script makes creating a lab much easier. If you are not an ADLDS expert, you can also save a lot of time and find a step by step instruction how to setup ADLDS fully automated.

Requirements

To install the lab, you need

A Hyper-V host running Windows 10 or Windows Server 2012 R2 or 2016 or an Azure subscription
The latest version of AutomatedLab

Expected Result

One domain controller for the contoso.com forest. The forest will have around 6.000 user accounts.
One Enterprise Certificate Authority in the contoso.com forest. The CA is used to issue certificates to the ADLDS instances to allow LDAPS communication.
4 servers that will host and replicate one ADLDS instance.
ADLDS syncs about 500 user accounts from AD.

The Deployment

VMs and basic services

To keep the deployment as simple as possible, AutomatedLab is used to deploy the virtual machines and the required services. Then some cmdlets of the AutomatedLab framework are used to deploy and configure ADLDS.

Note: AutomatedLab allows you to have all the code on your host computer and invoke the it on the VMs. It uses PowerShell remoting and adds more comfort to this standard feature. There is no need to care about credentials or double hope authentication issues.

[code lang="PowerShell"] $labName = 'ADLDS1' New-LabDefinition -Name $labName -DefaultVirtualizationEngine HyperV #defining default parameter values, as these ones are the same for all the machines $PSDefaultParameterValues = @{ 'Add-LabMachineDefinition:Memory' = 1GB 'Add-LabMachineDefinition:OperatingSystem' = 'Windows Server 2016 SERVERDATACENTER' 'Add-LabMachineDefinition:Network' = "$labName" } Add-LabVirtualNetworkDefinition -Name $labName -AddressSpace 192.168.40.0/24 $postInstallActivity = @() $postInstallActivity += Get-LabPostInstallationActivity -ScriptFileName PrepareRootDomain.ps1 -DependencyFolder $labSources\PostInstallationActivities\PrepareRootDomain $postInstallActivity += Get-LabPostInstallationActivity -ScriptFileName 'New-ADLabAccounts 2.0.ps1' -DependencyFolder $labSources\PostInstallationActivities\PrepareFirstChildDomain Add-LabMachineDefinition -Name aDC1 -Roles RootDC -DomainName contoso.com -PostInstallationActivity $postInstallActivity Add-LabMachineDefinition -Name aCA1 -Roles CaRoot -DomainName contoso.com Add-LabMachineDefinition -Name aLDS01 Add-LabMachineDefinition -Name aLDS02 Add-LabMachineDefinition -Name aLDS03 Add-LabMachineDefinition -Name aLDS04 Install-Lab $machines = Get-LabVM Install-LabWindowsFeature -ComputerName $machines -FeatureName RSAT-AD-Tools -AsJob Install-LabSoftwarePackage -ComputerName $machines -Path $labSources\SoftwarePackages\Notepad++.exe -CommandLine /S -AsJob Install-LabSoftwarePackage -ComputerName $machines -Path $labSources\SoftwarePackages\winrar.exe -CommandLine /S -AsJob Get-Job -Name 'Installation of*' | Wait-Job | Out-Null $machines = Get-LabVM | Where-Object Name -Like *lds* Install-LabWindowsFeature -ComputerName $machines -FeatureName ADLDS Checkpoint-LabVM -All -SnapshotName 1 Show-LabDeploymentSummary -Detailed [/code] After having run this code block, all the machines, the AD domain and enterprise CA are ready. The ADLDS feature is installed on all future ADLDS servers and a snapshot was created of the whole lab. The cmdlet Show-LabDeploymentSummary prints out a detailed summary about what has been deployed.

ADLDS Definitions

The next couple of lines are definitions that will be used later. You can modify names and ports if required. [code lang="PowerShell"] $allAdldsServers = Get-LabVM | Where-Object Name -Like *lds* $firstAdldsServer = $allAdldsServers | Select-Object -First 1 $otherAdldsServer = $allAdldsServers | Select-Object -Skip 1 $instanceName = 'LDS1' $instancePartitionName = "dc=$instanceName,dc=com" $instanceLdapPort = 5000 $instanceLdapsPort = 5001 $adamServiceUserName = 'AdldsService' $adamServiceUserPassword = 'Password1' $adamSyncUserName = 'AdamSync' $adamSyncUserPassword = 'Password1' $domainName = (Get-LabVM -Role RootDC).DomainName $domainDefaultNamingContext = Invoke-LabCommand -ComputerName (Get-LabVM -Role RootDC) -ScriptBlock { (Get-ADRootDSE).DefaultNamingContext } -PassThru $netBiosDomainName = $domainName.Split('.')[0] $sslCertificateSubject = "*.$domainName" [/code]

Requirements like DNS and Service Accounts

Part of the lab is also enabling LDAPS for the ADLDS instance. This requires accessing the ADLDS instance by the same DNS name used to issue the LDAPS certificate. This step creates the DNS entries on the domain controller so you can reach the ADLDS instance by alds01.contoso.com to alds04.contoso.com. [code lang="PowerShell"] Invoke-LabCommand -ActivityName 'Add DNS record for ADLDS Server' -ComputerName (Get-LabVM -Role RootDC) -ScriptBlock { $zone = Get-DnsServerZone -Name $env:USERDNSDOMAIN foreach ($adldsServer in $allAdldsServers) { Add-DnsServerResourceRecordA -Name $adldsServer.Name -IPv4Address $adldsServer.IpV4Address -ZoneName $zone.ZoneName } } -Variable (Get-Variable allAdldsServers) [/code] ADLDS will be configured to use a local service account. This service account must be created on all computers that will host the ADLDS instance. Username and password are used as defined previously. The service account does not have to be a local administrator. However, if the service user is not an administrator and want to use LDAPS, you have to assign read permissions to the service account on the certificate's private key stored in C:\Users\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys. [code lang="PowerShell"] Invoke-LabCommand -ActivityName 'Create Service User' -ComputerName $allAdldsServers -ScriptBlock { $password = $adamServiceUserPassword | ConvertTo-SecureString -AsPlainText -Force $user = New-LocalUser -Name $adamServiceUserName -Password $password Add-LocalGroupMember -Group Administrators -Member $user } -Variable (Get-Variable -Name adamServiceUserName, adamServiceUserPassword) [/code]

Installing ADLDS

The next code block installs the first ADLDS instance (InstallType=unique). Before starting the ADLDS installation an additional schema file is added to the ADAM directory. The schema file connects the auxiliary class 'msDS-BindableObject' to the user class to allow users authenticating against the ADLDS instance using LDAP simple bind. As the additional schema file is in the ADAM directory, it can be imported as part of the ADLDS installation. [code lang="PowerShell"] Invoke-LabCommand -ActivityName InstallADLDS -ComputerName $firstAdldsServer -ScriptBlock { $ldf = @' dn: CN=User,CN=Schema,CN=Configuration,DC=X changetype: Modify add: auxiliaryClass auxiliaryClass: msDS-BindableObject - dn: changetype: modify add: schemaUpdateNow schemaUpdateNow: 1 - '@ | Out-File -FilePath C:\Windows\ADAM\MS-adamschemaBindableObject.LDF <# The following text file defines the ADLDS installation. It is later used to adaminstall.exe as a parameter input file. This process is documented in [Perform an Unattended Install of an AD LDS Instance https://technet.microsoft.com/en-us/library/cc816774(v=ws.10).aspx] #> @" [ADAMInstall] InstallType=Unique InstanceName=$instanceName LocalLDAPPortToListenOn=$instanceLdapPort LocalSSLPortToListenOn=$instanceLdapsPort NewApplicationPartitionToCreate="$instancePartitionName" DataFilesPath=C:\Program Files\Microsoft ADAM\$instanceName\data LogFilesPath=C:\Program Files\Microsoft ADAM\$instanceName\data ImportLDIFFiles="MS-AdamSyncMetadata.ldf" "MS-adamschemaw2k8.LDF" "MS-User.ldf" "MS-adamschemaBindableObject.LDF" AddPermissionsToServiceAccount=Yes ServiceAccount=$adamServiceUserName ServicePassword=$adamServiceUserPassword "@ | Out-File -FilePath C:\adldsInstall.txt C:\Windows\ADAM\adaminstall.exe /answer:C:\adldsInstall.txt } -Variable (Get-Variable -Name instanceName, instancePartitionName ,instanceLdapPort, instanceLdapsPort, adamServiceUserName, adamServiceUserPassword) -PassThru [/code] Then the replicas can be installed using also the definitions made above. [code lang="PowerShell"] Invoke-LabCommand -ActivityName InstallADLDS -ComputerName $otherAdldsServer -ScriptBlock { <# The following text file defines the ADLDS installation. It is later used to adaminstall.exe as a parameter input file. This process is documented in [Perform an Unattended Install of an AD LDS Instance https://technet.microsoft.com/en-us/library/cc816774(v=ws.10).aspx] #> @" [ADAMInstall] InstallType=Replica InstanceName=$instanceName LocalLDAPPortToListenOn=$instanceLdapPort LocalSSLPortToListenOn=$instanceLdapsPort ApplicationPartitionsToReplicate="$instancePartitionName" SourceServer=$firstAdldsServer SourceLDAPPort=$instanceLdapPort DataFilesPath=C:\Program Files\Microsoft ADAM\$instanceName\data LogFilesPath=C:\Program Files\Microsoft ADAM\$instanceName\data AddPermissionsToServiceAccount=Yes ServiceAccount=$adamServiceUserName ServicePassword=$adamServiceUserPassword "@ | Out-File -FilePath C:\adldsInstall.txt C:\Windows\ADAM\adaminstall.exe /answer:C:\adldsInstall.txt } -Variable (Get-Variable -Name instanceName, instancePartitionName, instanceLdapPort, instanceLdapsPort, firstAdldsServer, adamServiceUserName, adamServiceUserPassword) -PassThru [/code]

Setting up LDAPS

Next step is to request an SSL certificate and put onto all ADLDS servers. The certificate must be in the ADLDS service account’s personal certificate store. [code lang="PowerShell"] $ca = Get-LabIssuingCA #the ADLDS VMs are not domain-joined so they cannot request certificates from the Enterprise CA. The LDAPS certificate #is requested on the CA itself... $cert = Request-LabCertificate -Subject "CN=$sslCertificateSubject" -OnlineCA $ca -TemplateName 'WebServer' -ComputerName $ca -PassThru #and then copied to the ASLDS servers Get-LabCertificate -ComputerName $ca -SearchString $sslCertificateSubject -FindType FindBySubjectName | Add-LabCertificate -ComputerName $allAdldsServers -ServiceName "ADAM_$instanceName" -Location CERT_SYSTEM_STORE_SERVICES [/code] To allow someone to connect to the ADLDS instance, a user is required. This code block creates an OU first, then creates a user account and adds this account to the Readers role. LDAP Simple Bind requires SSL, as the option 'RequireSecureSimpleBind' will be set to '1' in this code block as well. [code lang="PowerShell"] Invoke-LabCommand -ActivityName 'Create a user to bind to ADLDS' -ComputerName $firstAdldsServer -ScriptBlock { $PSDefaultParameterValues = @{ '*-AD*:Server' = "localhost:$instanceLdapPort" } $path = "DC=$instanceName,DC=com" $rootDse = Get-ADRootDSE Set-ADObject -Identity "CN=Directory Service,CN=Windows NT,CN=Services,$($rootDse.configurationNamingContext)" -Remove @{ 'msDS-Other-Settings' = 'RequireSecureSimpleBind=0' } Set-ADObject -Identity "CN=Directory Service,CN=Windows NT,CN=Services,$($rootDse.configurationNamingContext)" -Add @{ 'msDS-Other-Settings' = 'RequireSecureSimpleBind=1' } $ou = New-ADOrganizationalUnit -Name Users -Path $path -PassThru -ProtectedFromAccidentalDeletion $false $pass = 'Somepass1' | ConvertTo-SecureString -AsPlainText -Force $user = New-ADUser -Name BindTest -AccountPassword $pass -Enabled $true -Path $ou -PassThru $group = Get-ADGroup -Identity "CN=Readers,CN=Roles,$path" $group | Add-ADGroupMember -Members $user Write-Host "LDAP Simple Bind test user account is: '$($user.DistinguishedName)'" } -Variable (Get-Variable -Name instanceName) -PassThru [/code] Now ADLDS is deployed and ready to use. You can connect to the instance using LDAP and LDAPS, but a simple bind works only over LDAPS. Replications should work between all instances. You can get a list of all replicas and about the replication state using the following commands: [code lang="PowerShell"] dsmgmt "partition management" "connection" "connect to server localhost:5000" quit 'list nc replicas "dc=lds1,dc=com"' quit quit repadmin /replsum alds01:5000 [/code] LDP.exe is quite useful to test the connectivity, authentication and to take a look at the naming contexts and their content. You can use port 5000 to connect via LDAP or 5001 using LDAPS. The previously created user BindTest can be used to authenticate against the ADLDS instance using a LDAP simple bind. The result will be like this: [code] res = ldap_simple_bind_s(ld, 'CN=BindTest,OU=Users,DC=LDS1,DC=com', ); // v.3 Authenticated as: 'CN=BindTest,OU=Users,DC=LDS1,DC=com'. [/code] If you try this over an LDAP connection instead of LDAPS, you will get the following error: [code] res = ldap_simple_bind_s(ld, 'CN=BindTest,OU=Users,DC=LDS1,DC=com', ); // v.3 Error <49>: ldap_simple_bind_s() failed: Invalid Credentials Server error: 8009030C: LdapErr: DSID-0C0903D3, comment: AcceptSecurityContext error, data 202d, v3839 Error 0x8009030C The logon attempt failed [/code]

Setting up ADAMSync

The next section of this post enables ADAMSync. ADAMSync will be configured to sync all accounts that have the email address *.test.com. This code block takes some 500 random users from the AD domain and changes their email address to the domain test.com. Only these accounts will be synced as defined by the sync LDAP filter in the next step. [code lang="PowerShell"] Invoke-LabCommand -ActivityName 'Add test email address' -ComputerName (Get-LabVM -Role RootDC) -ScriptBlock { $rootDse = Get-ADRootDSE $users = Get-ADUser -Filter * -SearchBase "OU=Lab Accounts,$($rootDse.defaultNamingContext)" 1..500 | ForEach-Object { $u = $users | Get-Random $u | Set-ADObject -Replace @{ mail = "$($u.GivenName).$($u.SurName)@test.com" } } } [/code] ADAMSync needs to authenticate against the AD domain, hence a user account in AD is required. The username and password is taken from the definitions done previously. [code lang="PowerShell"] Invoke-LabCommand -ActivityName 'Create user for ADAMSync' -ComputerName (Get-LabVM -Role RootDC) -ScriptBlock { $password = $adamSyncUserPassword | ConvertTo-SecureString -AsPlainText -Force New-ADUser -Name $adamSyncUserName -AccountPassword $password -Enabled $true } -Variable (Get-Variable adamSyncUserName, adamSyncUserPassword) [/code] This XML document defines how the ADLDS sync should work. This definition is later stored in the partition to sync using 'adamsync.exe /install'. You can view the sync configuration stored in the property 'configurationFile' on the partition root defined in the sync ('target-dn'). This property can be viewed with ldp.exe or adsiedit.msc. The XML document contains the source and target information, logon information and an LDAP filter to select which objects should be synced.

NOTE: XML does not allow the usage of all characters. This can be an issue with the field object-filter 'adamsync.exe /install' does not trigger the sync, it does only define it.

[code lang="PowerShell"] Invoke-LabCommand -ComputerName $firstAdldsServer -ScriptBlock { @" Sync for $instanceName object $domainName $domainDefaultNamingContext $adamSyncUserName $netBiosDomainName

DC=$instanceName,DC=com

$domainDefaultNamingContext (&(|(objectClass=user)(objectClass=group)(objectClass=publicFolder))(mail=*@test.com)) surname givenName mail displayName name 5

"@ | Out-File -FilePath C:\MS-AdamSyncConf.xml "C:\Windows\ADAM\adamsync.exe /install localhost:$instanceLdapPort C:\MS-AdamSyncConf.xml /passPrompt" | Out-File -FilePath C:\InstallAdamSync.bat -Encoding ascii $cmd = "C:\Windows\ADAM\adamsync.exe /sync localhost:$instanceLdapPort '$instancePartitionName' /log C:\ADAMSyncLog{0:yyMMdd_hhmmss}.log" -f (Get-Date) $cmd = [scriptblock]::Create($cmd) $t = New-JobTrigger -RepetitionInterval (New-TimeSpan -Hours 1) -RepeatIndefinitely $true -At (Get-Date).AddMinutes(1) Register-ScheduledJob -Name AdldsSync -ScriptBlock $cmd -Trigger $t | Out-Null } -Variable (Get-Variable -Name domainName, instanceName, domainDefaultNamingContext, adamSyncUserName, netBiosDomainName, instanceLdapPort) -PassThru [/code]

NOTE: 'adamsync.exe /install' must be called manually, as the password prompt does not work within a script. This seems to be a shortcoming of ADAMSync.exe. The code block puts the install command into a batch file that you can call on the VM hosting the first ADLDS instance using PowerShell (do not use the ISE).

After ADAMSync is installed, you can start the sync by calling 'adamsync.exe /sync'. A scheduled task was created on the firest ADLDS instance that runs the sync every 1 hour. If you want to run it right after installing ADAMSync, you can use the following command to invoke it manually. [code lang="PowerShell"] Get-ScheduledJob -Name AdldsSync | Set-ScheduledJob -RunNow [/code] The scheduled task calling ADAMSync writes a log in C:\ that contains info about all synced objects.

Visualize DSC Reporting with PowerBI

Raimund Andrée — Sun, 21 May 2017 17:09:22 GMT

Abstract

Reporting is an essential component of DSC and one of the main advantages. Always knowing the status about your nodes and even being able to know which settings are not compliant is a big advantage compared to classical scripting or Group Policies. However, the reporting feature is not so easy to use and there are no graphical reports available yet. Dashboards and diagrams are always good to draw people’s attention. This article describes you to achieve exactly this. Having the graphical reports should also help explaining to non-technical people the importance of Infrastructure as Code and DSC.

Requirements

This article based on Using SQL Server 2016 for a DSC Pull Server. You need to move from EDB to an MDB with linked tables. All the steps required and additional details are mentioned in this article. The script to create the SQL database referenced in this article adds some views, functions and a trigger to convert the data into proper JSON, also required. Another requirement may be – depending on the size of the configuration – described in Broken DSC Reporting, RequestEntityTooLarge and some LCM internals. If you want to test this out without investing too much time, you may want to take a look at AutomatedLab and the script DSC Pull Scenario 1 (Pull Configuration, SQL Reporting).ps1 in paticular. This script deploys a lab that has a DSC Pull Server, SQL Server 2016 and PowerBI already configured.

Desired Result

The result after having implemented the reporting as described here looks like this: This dashboard lets you very easily see the overall status of your DSC nodes. In the current version, you get these information:

Node Status, count of success and failure
Count of compliant / non-compliant configuration items per node
Average duration it takes to apply the configuration
Status reports and configurations per node

The dashboard is quite extensible and can visualize almost anything that is covered with the data in the DSC database.

The heavy lifting behind the scenes

Note: This section is not essential to read of you just want to use the reporting solution and do not face issues.

The pull server stores most of the data as JSON into the database. However, the JSON data cannot be converted back from JSON so easy and seems to be serialized more than once. This is how the data looks like [code lang="powershell"] Invoke-Sqlcmd -Query 'SELECT TOP 1 StatusData FROM Statusreport' -Database DSC | Select-Object -ExpandProperty StatusData [/code]

["{\"StartDate\":\"2017-05-21T18:20:51.5710000+02:00\",\"IPV6Addresses\":[\"fe80::2c81:cc87:4f57:e1d9%2\",\"::2000:0:0:0\",\"::1\",\"::2000:0:0:0\"],\"DurationInSeconds\":\"5\",\"JobID\":\"{74703B6E-3E41-11E7-8365-0017FA00000C}\",\"CurrentChecksum\":\"30B2DD729B96A4DFCB882379448A9183C00B77EE754CCA5C84D76124AE16B151\",\"MetaData\":\"Author: Install; Name: PullClient; Version: 2.0.0; GenerationDate: 05/21/2017 18:04:02; GenerationHost: XPULL1;\",\"Status\":\"Success\",\"IPV4Addresses\":[\"192.168.88.8\",\"127.0.0.1\"],\"LCMVersion\":\"2.0\",\"NumberOfResources\":\"0\",\"Type\":\"LocalConfigurationManager\",\"HostName\":\"XDC13\",\"RebootRequested\":\"False\",\"MACAddresses\":[\"00-17-FA-00-00-0C\",\"00-00-00-00-00-00-00-E0\"],\"MetaConfiguration\":{\"AgentId\":\"1B664F0D-3E3F-11E7-8365-0017FA00000C\",\"ConfigurationDownloadManagers\":[{\"RegistrationKey\":\"\",\"ServerURL\":\"https://xpull1:8080/PSDSCPullServer.svc\",\"ResourceId\":\"[ConfigurationRepositoryWeb]PullServer\",\"ConfigurationNames\":[\"Complete_xDC13\"],\"SourceInfo\":\"C:\\\\Users\\\\Install.BMW\\\\Desktop\\\\DscScripts\\\\2.2 SetupDScClients.ps1::26::9::ConfigurationRepositoryWeb\"}],\"ActionAfterReboot\":\"ContinueConfiguration\",\"LCMCompatibleVersions\":[\"1.0\",\"2.0\"],\"LCMState\":\"Idle\",\"ResourceModuleManagers\":[],\"ReportManagers\":[{\"RegistrationKey\":\"\",\"ServerURL\":\"https://xpull1:8080/PSDSCPullServer.svc\",\"ResourceId\":\"[ReportServerWeb]ReportServer\",\"SourceInfo\":\"C:\\\\Users\\\\Install.BMW\\\\Desktop\\\\DscScripts\\\\2.2 SetupDScClients.ps1::34::9::ReportServerWeb\"}],\"StatusRetentionTimeInDays\":\"10\",\"LCMVersion\":\"2.0\",\"ConfigurationMode\":\"ApplyAndAutoCorrect\",\"RefreshFrequencyMins\":\"30\",\"RebootNodeIfNeeded\":\"True\",\"SignatureValidationPolicy\":\"NONE\",\"RefreshMode\":\"Pull\",\"DebugMode\":[\"NONE\"],\"LCMStateDetail\":\"\",\"AllowModuleOverwrite\":\"False\",\"ConfigurationModeFrequencyMins\":\"15\",\"SignatureValidations\":[]},\"Locale\":\"en-US\",\"Mode\":\"Pull\"}"]

PowerShell cannot convert this data using ConvertFrom-Json and SQL Server cannot read it either using OPENJSON. The data needs to be converted first. The solution was mentioned first in Get DSC Pull Client Status and is actually quite simple. To make the conversion happen completely transparent to the user and without manual interaction, I have created a database trigger that takes care of that. After the conversion, the StatusData column looks like this:

{"StartDate":"2017-05-21T18:57:28.2910000+02:00","IPV6Addresses":["fe80::e466:5405:663c:6ac3%3","::2000:0:0:0","::1","::2000:0:0:0"],"DurationInSeconds":"12","JobID":"{91CB841E-3E46-11E7-8365-0017FA00000D}","CurrentChecksum":"0FA45B9AE2E7CA252D64AA40DE8AFD1F9B19E02844F57498F22C1FD700F31E90","MetaData":"Author: Install; Name: Complete; Version: 2.0.0; GenerationDate: 05/21/2017 18:03:44; GenerationHost: XPULL1;","RebootRequested":"False","Status":"Success","IPV4Addresses":["192.168.88.9","127.0.0.1"],"LCMVersion":"2.0","NumberOfResources":"1","Type":"Initial","HostName":"XDC14","ResourcesInDesiredState":[{"SourceInfo":"C:UsersInstall.BMWDesktopDscScripts1.2 NewDscConfig.ps1::61::21::File","ModuleName":"PSDesiredStateConfiguration","DurationInSeconds":"0.019","InstanceName":"TestFile","StartDate":"2017-05-21T18:57:40.6890000+02:00","ResourceName":"File","ModuleVersion":"1.1","RebootRequested":"False","ResourceId":"[File]TestFile","ConfigurationName":"Complete","InDesiredState":"True"}],"MACAddresses":["00-17-FA-00-00-0D","00-00-00-00-00-00-00-E0"],"MetaConfiguration":{"AgentId":"1B67D9E9-3E3F-11E7-8365-0017FA00000D","ConfigurationDownloadManagers":[{"RegistrationKey":"","ServerURL":"https://xpull1:8080/PSDSCPullServer.svc","ResourceId":"[ConfigurationRepositoryWeb]PullServer","ConfigurationNames":["Complete_xDC14"],"SourceInfo":"C:UsersInstall.BMWDesktopDscScripts2.2 SetupDScClients.ps1::26::9::ConfigurationRepositoryWeb"}],"ActionAfterReboot":"ContinueConfiguration","LCMCompatibleVersions":["1.0","2.0"],"LCMState":"Idle","ResourceModuleManagers":[],"ReportManagers":[{"RegistrationKey":"","ServerURL":"https://xpull1:8080/PSDSCPullServer.svc","ResourceId":"[ReportServerWeb]ReportServer","SourceInfo":"C:UsersInstall.BMWDesktopDscScripts2.2 SetupDScClients.ps1::34::9::ReportServerWeb"}],"StatusRetentionTimeInDays":"10","LCMVersion":"2.0","ConfigurationMode":"ApplyAndAutoCorrect","RefreshFrequencyMins":"30","RebootNodeIfNeeded":"True","SignatureValidationPolicy":"NONE","RefreshMode":"Pull","DebugMode":["NONE"],"LCMStateDetail":"","AllowModuleOverwrite":"False","ConfigurationModeFrequencyMins":"15","SignatureValidations":[]},"Locale":"en-US","Mode":"Pull"}

Now the data can be just taken to PowerShell and converted with ConvertFrom-Json and also SQL Server is happy. When the LCM starts applying a configuration (Start-DscConfiguration), it writes an entry of type ‘Initial’ into the reporting database. This record has a StartTime but not yet a valid EndTime. Also, StatusData only contains some basic information about the node.

Note: The OperationType ‘Initial’ indicates that the node applies a configuration the first time. All subsequent applications are logged with the OperationType ‘Consistency’. The OperationType ‘LocalConfigurationManager’ logs the configuration of the LCM.

The trigger fires after an update happens. When the pull server send the StatusData to the database, backslashes are removed and the characters indicating an array in JSON. [code lang="sql"] CREATE TRIGGER [dbo].[DSCStatusReportOnUpdate] ON [dbo].[StatusReport] AFTER UPDATE AS SET NOCOUNT ON BEGIN DECLARE @JobId nvarchar(50) = (SELECT JobId FROM inserted); DECLARE @StatusData nvarchar(MAX) = (SELECT StatusData FROM inserted); IF @StatusData LIKE '\[%' ESCAPE '\' SET @StatusData = REPLACE(SUBSTRING(@StatusData, 3, Len(@StatusData) - 4), '\', '') DECLARE @Errors nvarchar(MAX) = (SELECT [Errors] FROM inserted); IF @Errors IS NULL SET @Errors = (SELECT Errors FROM StatusReport WHERE JobId = @JobId) IF @Errors LIKE '\[%' ESCAPE '\' AND Len(@Errors) > 4 SET @Errors = REPLACE(SUBSTRING(@Errors, 3, Len(@Errors) - 4), '\', '') UPDATE StatusReport SET StatusData = @StatusData, Errors = @Errors WHERE JobId = @JobId END [/code] After the conversion, SQL Server can parse and query the data in the column StatusData as JSON. This is the process of getting to the details in a JSON-formatted column.

How to retrieve data from a JSON column

This query is getting just the "StartDate" for all nodes. StartDate is a value in the JSON column "StatusData". [code lang="sql"] SELECT dbo.StatusReport.NodeName, JSON_VALUE([StatusReport].StatusData, '$.Status') FROM dbo.StatusReport [/code] or [code lang="sql"] SELECT dbo.StatusReport.NodeName,( SELECT [Value] FROM OPENJSON ([StatusData]) WHERE [Key] = 'StartDate' ) AS StartDate FROM dbo.StatusReport [/code] The functions “tvfGetNodeStatus” and “tvfGetRegistrationData” are for combining the data accessible directly using SQL and what is store as JSON. The queries in these functions are more complex, as there is JSON stored in JSON stored in SQL. These two functions can be access through the views “vRegistrationData” and “vNodeStatusComplex” and these views are used by PowerBI.

PowerBI - the interesting part

Starting with a new workbook

So, after having the database in place and some nodes registered and have them already send some feedback, it is time to make the data visible. After opening PowerBI you will see a welcome screen. There is an option named "Get data". After clicking on that option, you will be guided through the process of setting up the connection. Please select "SQL Server database", then you have to provide a server and database name and decide if you want to import the data or use DirectQuery. After finishing that you should see a list of all available sources. Please select all the views here (v*) and click on load. After finishing this step, you have to choose a visualization. Click on the Pie Chart and you get a grey template on your empty workbook. Keep the focus on the Pie Chart and in the Fields menu on the right side expand the node "vNodeStatusComplex". Please select from the field "Status" first and then "NodeName". This will move "Status" to "Legend" and "NodeName" to "Details". Obviously, this does not work. Please drag and drop the "NodeName" from "Details" to "Values". Voila, if all your DSC nodes are compliant, you have now a Pie Chart with just one big piece. In my environment, 2 out of 8 does have problems so it looks like this:

Using an existing workbook

If you want to get started quickly, you can download the PowerBI file Dashboard.pbix, the first screenshot has been taken from this file. You have to change the data source as your SQL Server pretty likely has a different name. Click on "Edit Queries -> Data source settings" and then on "Change Source".

Summary

After walking through this article, you have a database on SQL Server 2016 that presents the node status data in a way that can be visualized with PowerBI or other tools. This moves DSC on a new level and should makes it more appealing for IT managers.

Broken DSC Reporting, RequestEntityTooLarge and some LCM internals

Raimund Andrée — Sat, 20 May 2017 00:43:01 GMT

Abtract

After migrating a complex DSC configuration using partial configurations to composite resources, DSC reporting using SQL Server as described in Using SQL Server 2016 for a DSC Pull Server did not work any longer or was no longer reliable. This article is about troubleshooting this and learning something about Event ID 4260 and how the LCM sends data to the reporting database.

The change that leads to the issue

The configuration for a node in my case is quite complex and consists of at least 200 configuration items, and the number is growing. To make the management of the configuration easier and being able to manage dependency, I needed to split the configuration into sections. Recently I have learned that the way to split large DSC configurations by means of composite resources. Unfortunately, everything was build on partial configurations. The conversion was actually much easier than expected. Everything was working afterwards but one important part was missing: Reporting. Somehow the data send from the LCM to the pull server is bigger when using composite resources instead of partial configurations, even if the configuration that the client creates contains the same configuration items.

The effect on the database

After not seeing any new data in the report, first I have taken a look at the database. When the LCM starts applying a configuration (Start-DscConfiguration), it writes an entry of type 'Initial' into the reporting database. This record has a StartTime but not yet a valid EndTime. Also, StatusData only contains some basic information about the node.

Note: The OperationType 'Initial' indicates that the node applies a configuration the first time. All subsequent applications are logged with the OperationType 'Consistency'. The OperationType 'LocalConfigurationManager' logs the configuration of the LCM.

[code lang="PowerShell"] Invoke-Sqlcmd -Database DSC -Query 'SELECT TOP 1 * FROM StatusReport' [/code]

JobId                : d7352efe-3ce2-11e7-836e-0017fa00000a
Id                   : 77baf262-3ca1-11e7-8365-0017fa00000a
OperationType        : Initial
RefreshMode          :
Status               :
LCMVersion           : 2.0
ReportFormatVersion  : 2.0
ConfigurationVersion :
NodeName             : XDC11
IPAddress            : 192.168.88.129;127.0.0.1;fe80::99e2:71d5:6f53:7fe0%5;::2000:0:0:0;::1;::2000:0:0:0
StartTime            : 5/20/2017 12:31:03 AM
EndTime              : 12/30/1899 12:00:00 AM
Errors               : []
StatusData           : ["{\"IPV4Addresses\":[\"192.168.88.129\",\"127.0.0.1\"],\"CurrentChecksum\":\"0926E211C6D8617558
                       55CED942CB734CC94BF620EC6E4F8D1B9889F5B57ED6C7\",\"MACAddresses\":[\"00-17-FA-00-00-0A\",\"00-00
                       -00-00-00-00-00-E0\"],\"Type\":\"Initial\",\"HostName\":\"XDC11\",\"Locale\":\"en-US\",\"StartDa
                       te\":\"2017-05-20T09:31:03.9100000+11:00\",\"JobID\":\"{D7352EFE-3CE2-11E7-836E-0017FA00000A}\",
                       \"LCMVersion\":\"2.0\",\"IPV6Addresses\":[\"fe80::99e2:71d5:6f53:7fe0%5\",\"::2000:0:0:0\",\"::1
                       \",\"::2000:0:0:0\"]}"]
RebootRequested      : False
AdditionalData       : [{"Key":"OSVersion","Value":"{\"VersionString\":\"Microsoft Windows NT 10.0.14393.0\",\"ServiceP
                       ack\":\"\",\"Platform\":\"Win32NT\"}"},{"Key":"PSVersion","Value":"{\"CLRVersion\":\"4.0.30319.4
                       2000\",\"PSVersion\":\"5.1.14393.0\",\"BuildVersion\":\"10.0.14393.0\"}"}]

After the LCM has finished the job, it sends an update statement with the same JobId to the pull server. The pull server then forwards this update query to the database server. In this update statement the LCM provides all the missing details like Status, StatusData and the EndTime. In my case, this did not happen and the database was filling up with records of OperationType 'Initial' or 'Consistency' that did not have an EndTime. Additionally, all my nice report queries and PowerBI did not work anymore. One takeaway is that if something does not work with reporting, first take look at the DSC nodes operational log. I found an error with the ID 4260 there that was telling something about "RequestEntityTooLarge". Of course, you can also use PowerShell to find these event log entries using Get-WinEvent. [code lang="PowerShell"] Get-WinEvent -FilterHashTable @{ LogName = 'Microsoft-Windows-Dsc/Operational'; ID = 4260 } | Select-Object -First 1 | Format-List [/code]

TimeCreated  : 5/21/2017 6:32:10 AM
ProviderName : Microsoft-Windows-DSC
Id           : 4260
Message      : Job {03B1E172-3D93-11E7-836F-0017FA00000A} :
               Http Client 77BAF262-3CA1-11E7-8365-0017FA00000A failed for WebReportManager for configuration The
               attempt to send status report to the server https://xpull1:8080/PSDSCPullServer.svc/Nodes(AgentId='77BAF
               262-3CA1-11E7-8365-0017FA00000A')/SendReport returned unexpected response code RequestEntityTooLarge..

The problem are some limitations on the pull server's web configuration. After some research and testing I was confused by contradictory statements and resolutions. After trying without success for quite some time, I finally got it working by adding this into the web.config in 'C:\inetpub\PSDSCPullServer'. [code lang="xml"] [/code] Unfortunately the xWebConfigKeyValue does not yet support doing these settings when setting up the DSC pull server using DSC. As I do not want to make this a manual step that I can forget, I have added these lines into the pull server's setup routine. [code lang="PowerShell"] $path = 'C:\inetpub\PSDSCPullServer\web.config' $doc = [[xml]](Get-Content -Path $path) $serviceModelNode = ($doc | Select-Xml -XPath '//system.serviceModel').Node $bindingsNode = ($doc | Select-Xml -XPath '//system.serviceModel/bindings').Node if (-not $bindingsNode) { $bindingsNode = $doc.CreateElement('bindings') [void]$serviceModelNode.AppendChild($bindingsNode) } $webHttpBindingNode = ($doc | Select-Xml -XPath '//system.serviceModel/bindings/webHttpBinding').Node if (-not $webHttpBindingNode) { $webHttpBindingNode = $doc.CreateElement('webHttpBinding') [void]$bindingsNode.AppendChild($webHttpBindingNode) } $bindingNode = ($doc | Select-Xml -XPath '//system.serviceModel/bindings/webHttpBinding/binding').Node if (-not $bindingNode) { $bindingNode = $doc.CreateElement('binding') [void]$webHttpBindingNode.AppendChild($bindingNode) } $bindingNode.SetAttribute('maxBufferPoolSize', [int]::MaxValue) $bindingNode.SetAttribute('maxReceivedMessageSize', [int]::MaxValue) $bindingNode.SetAttribute('maxBufferSize', [int]::MaxValue) $bindingNode.SetAttribute('transferMode', 'Streamed') $doc.Save($path) [/code] After this change, it requires restarting the pull server's app pool. The easiest way is calling iisreset.exe.

Reporting works again

After having altered the web.config and restarted the app pool, the event ID 4260 on the DSC nodes got replaced by 4266, which looks much better. Also the records in the database are being updated again, as you can see by the valid EndTime.

Note: What is not visible in the screenshots is the column StatusData which stores the most interesting data.

Some Richer Reporting

If you create the database with the script references in SQL Server 2016 for a DSC Pull Server you have a view that parses the Json data and returns also some valuable information like Duration (how long has it taken applying the configuration) and how many resources are or are not in desired state. The non-compliant resources are also names in the column ResourceIdsNotInDesiredState. I hope this helps. Have fun with PowerShell and DSC.

PowerShell Tip: Getting enum values as names, int and bit

Raimund Andrée — Thu, 18 May 2017 16:44:16 GMT

There are a number of samples how to resolve enums and display all the possible values. However, none of those worked for me with the enum System.Security.AccessControl.FileSystemRights. This enum has duplicate values and as most functions listing enum values are based on Enum.GetValues, you see not just the duplicate values but also duplicate names. So while sitting in an airplane in Cologne waiting for the staff to fix some valve, i couldn’t think of a better way to spend my time than writing about this world-changing code snippet. To solve that, I have created this little helper function, an ideal candidate for your PowerShell profile. It uses Enum.GetNames hence shows the names as defined in the enum regardless if they have the same value. Get-Enum prints the binary values if you pass in a Flags enum. If the enum is not a binary one, the output will be only name and value. [code lang="powershell"] function Get-Enum { param ( [type]$Type ) if ($Type.BaseType.FullName -ne 'System.Enum') { Write-Error "Type '$Type' is not an enum" return } if ($Type.CustomAttributes | Where-Object { $_.AttributeType -eq [System.FlagsAttribute] }) { Write-Host "Type '$Type' is a Flags enum" Write-Host $isFlagsEnum = $true } $props = @( @{ Name = 'Name'; Expression={ [string]$_ } } @{ Name = 'Value'; Expression={ [uint32](Invoke-Expression "[$($type.FullName)]'$_'") }} ) if ($isFlagsEnum) { $props += @{ Name = 'Binary'; Expression={[Convert]::ToString([uint32](Invoke-Expression "[$($type.FullName)]'$_'"), 2)}} } [enum]::GetNames($Type) | Select-Object -Property $props } [/code] You can call the function like this [code lang="powershell"] Get-Enum System.DayOfWeek [/code] to get this output: [code] Name Value ---- ----- Sunday 0 Monday 1 Tuesday 2 Wednesday 3 Thursday 4 Friday 5 Saturday 6 [/code] And this is the example of the enum that made me write this post [code lang="powershell"] Get-Enum -Type System.Security.AccessControl.FileSystemRights [/code] And as this is a flags enum, the function displays the binary representation of the value as well: [code] Name Value Binary ---- ----- ------ ListDirectory 1 1 ReadData 1 1 WriteData 2 10 CreateFiles 2 10 CreateDirectories 4 100 AppendData 4 100 ReadExtendedAttributes 8 1000 WriteExtendedAttributes 16 10000 Traverse 32 100000 ExecuteFile 32 100000 DeleteSubdirectoriesAndFiles 64 1000000 ReadAttributes 128 10000000 WriteAttributes 256 100000000 Write 278 100010110 Delete 65536 10000000000000000 ReadPermissions 131072 100000000000000000 Read 131209 100000000010001001 ReadAndExecute 131241 100000000010101001 Modify 197055 110000000110111111 ChangePermissions 262144 1000000000000000000 TakeOwnership 524288 10000000000000000000 Synchronize 1048576 100000000000000000000 FullControl 2032127 111110000000111111111 [/code] Have fun!

Using SQL Server 2016 for a DSC Pull Server

Raimund Andrée — Thu, 11 May 2017 11:29:51 GMT

Abstract

This article describes how to use a SQL Server 2016 as the backend database for a Desired State Pull Server. The default database engine is EDB and lacks a couple of feature required. It is quite hard to create reports based on the EDB and using the web service to retrieve data can get extremely slow. This is why we needed another solution for DSC reporting. This article guides you through the setup process.

Credits

This article is inspired by Using SQL Server DB for DSC and adds more automation and a better security model. Also worth a look the post Use SQL Server as DSC Pull Server Backend?. It explains why you cannot use SQL Server directly and have to go over the MDB.

Requirements

You need to have a good understanding about DSC in general and how to setup a DSC pull server. To be able to successfully execute all the steps described here, you need a domain joined DSC pull server and a SQL Server 2016 joined to the same forest. One or more machines that will become DSC nodes are also quite helpful. If you want to setup the requirements automatically, look at AutomatedLab and the script DSC Pull Scenario 1 (Pull Configuration).ps1. Only the SQL Server 2016 is missing in this deployment script but can be added with just to lines: [code lang="powershell"] Add-LabIsoImageDefinition -Name SQLServer2016 -Path $labSources\ISOs\en_sql_server_2016_standard_with_service_pack_1_x64_dvd_9540929.iso Add-LabMachineDefinition -Name DSQL1 -Roles SQLServer2016 [/code] If you want to redeploy the scenario that this article describes, you can use the script DSC Pull Scenario 1 with SQL Server.ps1. It takes care of the requirements plus all what is described in this post fully automated. So, if you just need a lab with a DSC pull server that stores all data on SQL, this can save you a huge amount of time.

Setting things ready on the SQL Server and connecting the pull server

On the SQL Server, you need to create the database for DSC and login for the computer account.

Note: The article mentioned in the credits section is great and saved me a lot of time. But one thing I did not like is providing credentials if not absolutely required. The initial approach used credentials stored along with the OleDB connecting in the registry. The setup described here works differently and gives the computer account of the DSC pull server write access to the database.

Creating the DB on the SQL Server

Note: All tasks in this section are executed on the SQL Server

The SQL script to create the database is published on CreateDscDatabase.ps1. It asks for a domain name and a computer name, creates a SQL Server login for that user and gives this login write access to the DSC database. The table setup is almost like described in Using SQL Server DB for DSC. There is only one additional column called AdditionalData in the table StatusReport. Along with the tables there is a trigger to transform the data into readable JSON and a couple of functions and views to get a report of your DSC nodes. Details about this will be covered in the next article. Please just run the script on the SQL Server like this by providing the domain name and the computer account name of the pull server. [code lang="powershell"] C:\CreateDscDatabase.ps1 -DomainName contoso -ComputerName dpull1 [/code] The output should look like this: [code] Creating the DSC database on the local default SQL instance...finished. Database is stored on C:\DSCDB [/code]

Connecting the DSC Pull Server with the SQL database

Note: All tasks in this section are executed on the Pull Server

The pull server must be configured to use MDB instead of the default EDB database

If not already done, please change the database model for the pull server from edb to mdb. If you deploy the pull server with AutomatedLab and the script mentioned earlier, the database model is already set to MDB. If you want to configure the pull server by yourself or want to change the database model after the deployment, you may want to use this DSC config file: SetupDscPullServerMdb.ps1. If everything is configured correctly, you should see these entries in the C:\inetpub\PSDSCPullServer\web.config file: [code lang="xml"] [/code]

ODBC Data Source

The MDB that we will use in the next step requires an OleDB connection named "DSC" which needs to point to the SQL Server that you have created the DSC database previously. You can create the data source either using the ODBC Data Source Administrator or PowerShell. The command for that is: [code lang="powershell"] Add-OdbcDsn -Name DSC -DriverName 'SQL Server' -Platform '32-bit' -DsnType System -SetPropertyValue @('Description=DSC Pull Server', "Server=", 'Trusted_Connection=yes', 'Database=DSC') -PassThru [/code]

Replacing the MDB with a version that is using "linked tables"

After changing the database model to MDB, the pull server writes all information into that file. However, the purpose of this article is to have the data in a SQL database. To achieve that, the MDB (C:\Program Files\WindowsPowerShell\DscService\Devices.mdb) should be replaced by this version Devices.mdb. This MDB uses linked tables to redirect the data to the SQL Server. If you are not sure what happens inside the file, just open it with Access and you will see the tables but no data yet on the right side, as there is nothing in the DSC database on the SQL Server.

Note: If you feel uncomfortable using the mdb that someone else uploaded, you can very easily create the file by yourself. The steps are described in Using SQL Server DB for DSC.

Testing the setup

This section covers a very easy test configuration and configures some machines to pull the configuration from the pull server. In order to do that data has to be written in the SQL database.

Defining the configuration

The configuration is just making sure that the given file has the content "OK". The node name is "localhost" so it applies to each machine that pulls it. [code lang="powershell"] Configuration 'TestConfig' { Import-DscResource –ModuleName 'PSDesiredStateConfiguration' Node localhost { File 'TestFile' { Ensure = 'Present' Type = 'File' DestinationPath = 'C:\DscTestFile' Contents = 'OK' } } } [/code]

Converting the configuration into a MOF

The local configuration manager cannot use DSC configuration written in PowerShell directly. You have to build a MOF first. [code lang="powershell"] TestConfig -OutputPath C:\DscTestConfig | Out-Null [/code] The MOF file always get the name of the given node, in this case "localhost.mof". This is not the best name for the configuration so the file is renamed to "TestConfig.mof". [code lang="powershell"] Rename-Item -Path C:\DscTestConfig\localhost.mof -NewName Rename-Item -Path C:\DscTestConfig\localhost.mof -NewName TestConfig.mof [/code]

Publishing the MOF

Now you are finished creating the MOF. But the pull server does not know about it hence clients cannot pull the MOF. Publishes configurations (MOFs) need to be in the folder C:\Program Files\WindowsPowerShell\DscService\Configuration along with a checksum. The easiest way to achieve this is by calling the following cmdlet. [code lang="powershell"] Publish-DSCModuleAndMof -Source C:\DscTestConfig [/code]

Telling the DSC nodes to pull the TestConfig

So far, no computer in the lab knows about the pull server. To introduce the pull server to the nodes, you must configure the local configuration manager, of course also using DSC. The following configuration does all this.

Note: The attribute DSCLocalConfigurationManager indicates that a special kind of configuration called meta configuration should be created. Meta configurations are configuring the LCM and have the extension .meta.mof. More info on this can be found in Configuring the Local Configuration Manager.

[code lang="powershell"] [DSCLocalConfigurationManager()] Configuration DscClientConfig { param( [Parameter(Mandatory)] [string]$PullServer, [Parameter(Mandatory)] [string]$RegistrationKey, [Parameter(Mandatory)] [string[]]$ComputerName, [Parameter(Mandatory)] [string]$ConfigurationName ) Node $ComputerName { Settings { RefreshMode = 'Pull' RefreshFrequencyMins = 30 ConfigurationModeFrequencyMins = 15 ConfigurationMode = 'ApplyAndAutoCorrect' RebootNodeIfNeeded = $true } ConfigurationRepositoryWeb PullServer { ServerURL = "https://$($PullServer):8080/PSDSCPullServer.svc" RegistrationKey = $RegistrationKey ConfigurationNames = $ConfigurationName } ReportServerWeb ReportServer { ServerURL = "https://$($PullServer):8080/PSDSCPullServer.svc" RegistrationKey = $RegistrationKey } } } [/code] Now this meta configuration needs to be built and finally pushed to the DSC nodes. The following code does this for all the computers that are in the computers container in Active Directory. The registration key is a GUID you have to generate and provide when setting up the pull server. This GUID is written to a text file that is required for the next step. [code lang="powershell"] $computerContainerDn = "CN=Computers,$((Get-ADRootDSE).defaultNamingContext)" $computerName = (Get-ADComputer -SearchBase $computerContainerDn -Filter *).DnsHostName $registrationKey = Get-Content -Path 'C:\Program Files\WindowsPowerShell\DscService\RegistrationKeys.txt' DscClientConfig -OutputPath c:\DscClientConfig -PullServer $env:COMPUTERNAME -RegistrationKey $registrationKey -ComputerName $computerName -ConfigurationName TestConfig Set-DscLocalConfigurationManager -Path C:\DscClientConfig -ComputerName $computerName Start-DscConfiguration -UseExisting -ComputerName $computerName -Wait -Verbose [/code]

Verifying the data on the SQL Server

Now it is time to open the SQL Server Management Studio on the SQL Server and navigate to the DSC database, then to the table "RegistrationData". Do a right-click and then click on "Select Top 1000 Rows". You should see data like this indicating that everything works as expected.

Conclusion

It looks more complicated as it is to move the DSC data to a SQL Server. Still, this is a workaround and I hope that in the future we can provide a connection string for SQL Server in the web.config directly (dbconnectionstr). Having the data now on SQL gives you a universe of tools to do reporting and data hygiene tasks. In this article, I have used SQL Server 2016 as it natively supports JSON which is important as the pull server stores JSON in SQL columns. The next article will cover advanced reporting with the data you have in SQL Server now.

Privileged Access Management - demystified

Michael Frommhold MSFT — Tue, 09 May 2017 21:37:05 GMT

Today's topic:

Privileged Access Management (PAM)

Just in Time Administration

demystified

Coming with Microsoft Server 2016, we offered a new optional feature for Active Directory: the 'Privileged Access Management Feature'. The new feature is only available with Domain Controllers OS >= Windows Server 2016. Privileged Access Management Feature consists of two parts:

Privileged Access Management (PAM) – aka Privileged Identity Management (PIM)
Just in Time Administration (JiTA)

PAM represents an extra layer of security for administration – basically it’s a sort of ‘security by obscurity’ in its most positive interpretation. How does it work (conceptually): After implementing a single domain forest as admin forest, trusted by your production forest, you create shadow principals in the admin forest, that bear the SIDs of administrative groups from the production forest with it, for example from the Enterprise Admins and the Domain Admins. You may then add user objects from the admin forest to the member attribute of the shadow principals. If such an user object logs on to resource in the production forest it will carry the SIDs of the administrative groups that correspond to the shadow principals user object is member of. Note: All types of security principals my be mirrored (-> all objects that have an objectSid). One of the nice effects is that you can call DCpromo from GUI with an account from your admin forest that has the production Domain Admins shadow group membership and will therefore successfully pass the DCPromo internal check for Domain Admins group membership. This happens because the Key Distribution Center (KDC) service on a Domain Controller calculates the SID list of an user’s group memberships provided by the forest this DC is member of - from:

sIDHistory
direct group memberships ( + the groups sIDHistory)
nested group memberships ( + the groups sIDHistory)
primaryGroupID ( –> member of = )
some default SIDs
memberships in shadow principals (the corresponding SIDs of the shadowed groups) This part only happens on DCs with OS >= Windows Server 2016.

If the trust created between production and admin forest has the following trust attributes bits set

enable sIDHistory
enable PIM trust

, the SIDs from the production forest in the calculated token make their way through the trust and thus will be present in the logon token in production. Saying – you utilize user accounts from your admin forest, that 'pretend' to be member of the Domain Admins or Enterprise Admins from the production forest, but they never where, are not or will not be actual member of the production Domain Admins or Enterprise Admins. This means that it is not possible to identify these user accounts as high-level admin accounts by querying the well known high-level administrative groups –> 'security by obscurity' :-) You may assign user accounts to the member attribute of groups or shadow principals permanently or temporary for a defined time span. If you assign membership temporary –> this is where Just in Time Administration joins the game. Note: JiTA is not limited to be only used with shadow principals over a PIM trust – you may use this in your forest / domain with normal group memberships as well. A temporary membership is flagged with an expiration time stamp, defined by the time of assigning the membership plus the time span defined for the membership. The Active Directory service (NTDS) on a Domain Controller with OS >= Windows Server 2016 inspects memberships and removes these memberships after expiration time is exceeded. Additionally does the Key Distribution Center (KDC) on such a DC set the issued Ticket Granting Ticket’s (TGT) End Time to the earliest expiration time of a membership (if there is any membership with expiration time present for the user). Session tickets will have the same End Time set. Saying – if the expiration time of a membership exceeded –>

the user’s membership to this group or shadow principal is revoked
the TGT and the session tickets have expired
user requests new TGT and session tickets when contacting the resources again –> the membership is not in the token anymore

How does it work (technically): Mandatory prerequisites are:

the DCs in the forest where you want to utilize JiTA / PAM are Windows Server 2016 (or higher) machines
Forest Functional Level and Domain Functional Level of the admin forest must be of 'Windows Server 2016'
for PAM usage:
- a forest trust between production forest and admin forest (production trusts admin)
- production forest DCs must be at least Windows Server 2012 R2 machines
- if production forest DCs are Windows Server 2012 R2 machines must have May 2016 Update Rollup installed – easy check: run netdom trust /? if you see /EnablePIMTrust as last help topic –> the patch is there.
- trustAttributes bit TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL (enable sIDHistory) is set in production for this trust
- trustAttributes bit TRUST_ATTRIBUTE_PIM_TRUST (enable PIM trust) is set in production for this trust
- optional feature 'Privileged Access Management Feature' is enabled in admin forest

The shadow principal object must be of objectClass msDs-ShadowPrincipal. The SID of the administrative group in your production forest is stored in the msDs-ShadowPrincipalSid attribute of the shadow principal. Mandatory attributes while creation of a msDs-ShadowPrincipal object are:

cn
msDs-ShadowPrincipalSid

Such shadow principals can only be created as child objects of a msDs-ShadowPrincipalContainer object. Shadow principals must be created in the default shadow principal container ' Shadow Principal Configuration', located in the Services container, child of the configuration naming context of your admin forest – exemplary path for contoso.com: CN=Shadow Principal Configuration,CN=Services,CN=Configuration,DC=contoso,DC=com Note: Even though you may create msDs-ShadowPrincipalContainer in your domain naming context and therefore be able to create shadow principals here – they will not be functional for the PAM purpose. To create a shadow principal I recommend to use PowerShell (PoSh) – the standard AD-Cmdlets are sufficient to perform the needed tasks. You will find sample code below. Adding an user account to a shadow principal is a standard LDAP add modification request for the member attribute –> AddRequest(distinguishedName of the shadow principal,member=distinguishedName of the user) If we want to add an user account for a defined time span only – we use a little extension in the add modification request –> AddRequest(distinguishedName of the shadow principal,member=time span in seconds,distinguishedName of the user>) There is PoSh sample code below. Note: If you update an already assigned temporary membership, LDAP will not throw an 'already in group' exception, but just calculate the new expiration time for the entry. How it's stored in the Active Directory database: As a quick starter: We do not store data by its data base path (distinguishedName) but by an incrementing numeric unique identifier. These identifiers, which are called Distinguished Name Tags (DNTs), are not replicated or otherwise visible to other domain controllers. As objects are created (directly or from replication) on a domain controller, a unique value is used. The table below shows the data tables in the AD database and their usage.

Table	Tree	Data	Usage
datatable		column, index	objects -> attributes
datatable	long_table	column, index	big data attributes
hiddentable		column	like system table in SQL
link_table		column, index	linked attribute pair assoc
link_history_table		column, index	linked attribute pair assoc history
sd_table		column, index	single instance storage for Security Descriptors
sdproptable		column, index	sd_table properties
sdpropcounttable		column	sd_table properties count
quota_table		column, index	AD quotas
Quota_rebuild_progress_table		column	AD quota progress

Structure of the datatable – for contoso.com:

DNT	Parent DNT	RelativeDN	RDNType
0	null	com	DC
1	0	contoso	DC
165	1	Configuration	CN
212	165	Services	CN
287	212	Shadow Principal Configuration	CN
2721	1	GoldAdmins	OU
3436	287	ShadowDA.production	CN
3445	2721	GoldAdmin.production	CN

The distinguishedName of an object is calculated by following the Parent DNT chain and combining RDNType and RelativeDN– examples:

User account to be used for administration in production forest: CN=GoldAdmin.production,OU=GoldAdmins,DC=contoso,DC=com
- DNT = 3445 –> DN: CN=GoldAdmin.production
- PDNT = 2721 –> DN: CN=GoldAdmin.production,OU=GoldAdmins
- PDNT = 1 –> DN: CN=GoldAdmin.production,OU=GoldAdmins,DC=contoso
- PDNT = 0 –> DN: CN=GoldAdmin.production,OU=GoldAdmins,DC=contoso,DC=com
Shadow principal shadowing production Domain Admins: CN=ShadowDA.production,CN=Shadow Principal Configuration,CN=Services,CN=Configuration,DC=contoso,DC=com
- DNT = 3436 –> DN: CN=ShadowDA.production
- PDNT = 287–> DN: CN=ShadowDA.production,CN=Shadow Principal Configuration
- PDNT = 212–> DN: CN=ShadowDA.production,CN=Shadow Principal Configuration,CN=Services
- PDNT = 165–> DN: CN=ShadowDA.production,CN=Shadow Principal Configuration,CN=Services,CN=Configuration
- PDNT = 1 –> DN: CN=ShadowDA.production,CN=Shadow Principal Configuration,CN=Services,CN=Configuration,DC=contoso
- PDNT = 0 –> DN: CN=ShadowDA.production,CN=Shadow Principal Configuration,CN=Services,CN=Configuration,DC=contoso,DC=com

The member attribute is part of a linked attribute pair in Active Directory. Linked attributes are pairs of attributes in which the system calculates the values of one attribute (the back link) based on the values set on the other attribute (the forward link) throughout the forest. A back-link value on any object instance consists of the distinguishedNames (DN) of all the objects that have the object's DN set in the corresponding forward link. The attributes 'member' and 'memberOf' are a pair of linked attributes, where 'member' is the forward link and 'memberOf' is the back link. The link is identified by the linkID values of two attributeSchema definitions:

The linkID of the forward link is an even, positive, nonzero value.
The back link linkID is the forward linkID plus one.

The LinkID for 'member' is 2 and the linkID for 'memberOf' is 3. Linked attribute data association is stored in the link table. We do not store the distinguishedName here, but the DNTs. Structure of the link table – exemplary showing membership of GoldAdmin.production (DNT 3445) in ShadowDA.production (DNT 3436), with naming context DNT (NC_DNT) for the configuration naming context (DNT 165) and an expiration time set to 2017-05-09 06:09:17:

link_DNT	link_base	backlink_DNT	NC_DNT	expiration_time
3436	member	3445	165	2017-05-09 06:09:17

Column expiration_time is the place where NTDS is checking for necessary membership revocation and KDC for earliest expiration time of an user's membership for TGT End Time calculation. Note: If the expiration time is exceeded, the entry in the link_table does not get deleted - it is just treated as invalid and therefore not returned from NTDS when querying group membership. Just for completeness - NTDS follows the link_DNT –> backlink_DNT chain to calculate direct and nested group membership – example:

userX (DNT 5610) is member of group2 (DNT 5239)
group2 (DNT 5239) is member of group1 (DNT 5112)

-> userX is member of group2 and group1

link_DNT	link_base	backlink_DNT	NC_DNT	expiration_time
5112	member	5239	1
5239	member	5610	1

How do the SIDs in the token come together: When we use all the above for Domain Admin access to a Domain Controller in production by a user from the admin forest, the following happens authentication-wise:

user tries to RDP to resource in production -> resource does not know this user -> needs authentication and authorization
authentication request gets redirected to a KDC (Key Distribution Center -> DC) in admin forest
KDC builts SID list like described above, including the mirrored SID of the Domain Admins from production
DC in production checks group nesting in his database for the SIDs presented from the token issued by the admin forest
production DC adds group nesting accordingly to the token !NOTE! : since security principals from trusted forests can only be nested in domain / builtin local groups of the trusting forest -> group nesting of the mirrored security principal in production will only be applied to the resulting token from groups that are of group type domain local or builtin local. Nesting in domain global or domain universal groups will not be added to the resulting token.

OK – this were the basics – now to another fun part – the coding. PoSh code sample to create the PAM trust between production and admin forest: This code must be run in the production forest with Domain Admin privileges. It creates the unidirectional forest trust between production and admin forest where production trusts admin forest. Mandatory command line argument:

–ForestName 'FDQN of the admin forest'

To create the trust we use the following classes and methods from the System.DirectoryServices.ActiveDirectory (S.DS.AD) namespace:

class DirectoryContext
class Forest
method Forest::CreateTrustRelationship

An overview of the available classes and their wrappings in .Net to communicate with Active Directory: Unfortunately the manipulations to trustAttributes flag is not possible per S.DS.AD implementation or direct LDAP modification, we must use some APIs – which is definitely no big fun in PoSh – therefore we call netdom.exe twice to enable sIDHistory and PIM trust for this trust on the trusting side (production). More details in the code comments. [code lang="powershell"] #region params param ( [Parameter(Position=0,Mandatory=$true)] [string]$ForestName ) #endregion #region functions #region retreive a DC in current forest in closest site #endregion function GetDC([string]$forestName) { [string]$ret = [String]::Empty $dc = $null if ($forestName -ne $null) { $dc = Get-ADDomainController -DomainName $forestName -NextClosestSite -Discover } else { $dc = Get-ADDomainController -NextClosestSite -Discover } $ret = $dc.HostName.Value return $ret } #region create forest trust # against target forest with given trustdirection and credentials #endregion function CreateForestTrust([string]$targetForest, ` [System.DirectoryServices.ActiveDirectory.TrustDirection]$direction, ` [pscredential]$targetUser) { [bool]$created = $false # get a DC in current forest $dcinfo = GetDC try { # initiate.Net context object from retrieved DC [System.DirectoryServices.ActiveDirectory.DirectoryContext]$ctx = ` New-Object System.DirectoryServices.ActiveDirectory.DirectoryContext('DirectoryServer', $dcinfo) &amp;nbsp; # initiate .Net forest object from context [System.DirectoryServices.ActiveDirectory.Forest]$forest = ` [System.DirectoryServices.ActiveDirectory.Forest]::GetForest($ctx) # initiate .Net context object for target forest [System.DirectoryServices.ActiveDirectory.DirectoryContext]$tctx = ` New-Object System.DirectoryServices.ActiveDirectory.DirectoryContext('Forest', ` $targetForest, ` $targetUser.UserName, ` $targetUser.GetNetworkCredential().Password) # initiate .Net forest object for target forest from context [System.DirectoryServices.ActiveDirectory.Forest]$target = ` [System.DirectoryServices.ActiveDirectory.Forest]::GetForest($tctx) $created = $true } catch { Write-Host "Failed to initialize necessary .Net classes : $($_.Exception)" } if ($created -eq $true) { $created = $false try { # create foresttrust $forest.CreateTrustRelationship($target, $direction) $created = $true Write-Host "Created trust between $targetForest and $($forest.Name)" } catch { Write-Host "Failed to create trust between $targetForest and $($forest.Name) : $($_.Exception)" } } if ($created -eq $true) { # Unfortunately we canot edit trustAttributes by LDAP operation -&amp;gt; # we must use LSA APIs -&amp;gt; no big fun in PoSh. # Therefore we call netdom.exe here- # enable sidhistory Write-Host "Setting TRUST_ATTRIBUTE_TREAT_AS_EXTERNAL (sIDHistory) trustAttributes bit" netdom trust "$($forest.Name)" "/Domain:$targetForest" /EnableSidhistory:Yes # enable PAM trust Write-Host "Setting TRUST_ATTRIBUTE_PIM_TRUST (PAM)&amp;nbsp;trustAttributes bit" netdom trust "$($forest.Name)" /Domain:$targetForest /EnablePIMTrust:Yes } } #endregion #region main [string]$targetforest = $ForestName # get target forest admin credentials [PSCredential]$targetuser = Get-Credential -Message "Provide administrative creds for $targetforest" # trust direction InBound - we trust the red (bastion) forest [System.DirectoryServices.ActiveDirectory.TrustDirection]$trustdir = ` [System.DirectoryServices.ActiveDirectory.TrustDirection]::Outbound CreateForestTrust $targetforest $trustdir $targetuser #endregion [/code] PoSh code sample to create a shadow principal in the admin forest, shadowing a group (default Domain Admins – RID = 512) from the production forest: This PoSh must be run in the admin forest with Domain Admin privileges of the forest root domain or Enterprise Admin privileges. Mandatory command line argument:

–ProdForest 'FDQN of the production forest'

Optional command line arguments:

–ProdGroupRid 'RID of the production forest group to shadow' (default = 512 –> Domain Admins)
–ForceFeature '$true / $false' (default = $true–> calls Enable-ADOptionalFeature 'Privileged Access Management Feature')

In this sample code we mainly use PoSh AD-Cmdlets – only the casting from the Domain Admin's objectSid to a value that can be used to be set in msDs-ShadowPrincipalSid attribute of the shadow principal is done with .Net (System.Security.Principal.SecurityIdentifier::GetBinaryForm). Workflow:

get domain SID of production forest and append given group RID (-> the SID to be used in msDs-ShadowPrincipalSid)
get default msDs-ShadowPrincipalContainer from configuration naming context of admin forest
[Optional] enable PAM feature
create shadow principal in default msDs-ShadowPrincipalContainer with the SID of the production forest group to shadow

More details in the code comments. [code lang="powershell"] #region params param ( # the production forest to contact for it's domain sid [Parameter(Mandatory=$true)] [string]$ProdForest, # the RID of the production forest group to shadow [Parameter(Mandatory=$false)] [string]$ProdGroupRid = "512", # should we call Enable-ADOptionalFeature Privileged Access Management Feature ? [Parameter(Mandatory=$false)] [bool]$ForceFeature = $true ) #endregion #region functions #region retrieve default msDs-ShadowprincipalContainer # from configuration naming context #endregion function GetShadowPrincipalContainer() { [string] $ret = "CN=Shadow Principal Configuration,CN=Services," # get a DC from current forest $dc = Get-ADDomainController -NextClosestSite -Discover # send UDP pipng to port 389 -&amp;gt; rootDSE call $rootDSE = Get-ADRootDSE -Server $dc.HostName.Value # read configurationNamingContext attribute $ret += "$($rootDSE.configurationNamingContext)" return $ret } #region get SID of the production forest group to shadow # we ask for production forest domain sid and append given group RID #endregion function GetGoldGroupSid([string]$goldForest, [string]$groupRid) { [byte[]] $ret = $null # get a DC in production forest $dc = Get-ADDomainController -DomainName $goldForest -NextClosestSite -Discover # send UDP pipng to port 389 -&amp;gt; rootDSE call $rootdse = Get-ADRootDSE -Server $dc.HostName.Value # read defaultNamingContext attribute -&amp;gt; domain naming context [string] $defnc = $rootdse.defaultNamingContext # get domain object - calling for objectSid attribute $dom = Get-ADObject $defnc -Server $dc.HostName.Value -Properties objectSid # build goup SID [string]$sddl = "$($dom.objectSid.Value)-$groupRid" # cast to SecurityIdentifier .Net object [System.Security.Principal.SecurityIdentifier] $sid = [System.Security.Principal.SecurityIdentifier]$sddl Write-Host "Gold SID: $($sid.Value)" # resize byte array $ret = New-Object byte[] $sid.BinaryLength # tranlsate SecurityIdentifier .Net object to byte array $waste = $sid.GetBinaryForm($ret, 0) return $ret } #region enable Privileged Access Management Feature (in red forest -&amp;gt; W2K16 AD) #endregion function EnableFeature() { try { Enable-ADOptionalFeature "Privileged Access Management Feature" ` –Scope ForestOrConfigurationSet ` -Target $redforest } catch { } } #endregion #region main # Enable-ADOptionalFeature can be skipped by passing -ForceFeature $false if ($ForceFeature -eq $true) { EnableFeature } # get Domain Admins Sid from gold [byte[]] $bsid = GetGoldGroupSid $ProdForest $ProdGroupRid # get path to CN=Shadow Principal Configuration,CN=Services,configNC [string] $path = GetShadowPrincipalContainer Write-Host "Shadow Principal Configuration: $path" # build shadow principal name for Gold group [string] $goldshadow = "GoldShadow$ProdGroupRid.$($ProdForest)" Write-Host "Creating $goldshadow" try { # create shadow principal New-ADObject -Type 'msDs-ShadowPrincipal' ` -OtherAttributes @{ 'msDs-ShadowPrincipalSid' = $bsid } ` -Name $goldshadow ` -Path $path Write-Host "Created $goldshadow" } catch { Write-Host "Failed to create $goldshadow ($($_.Exception.Message))" } #endregion [/code] PoSh code sample to (temporarely) nest an account from the admin forest into a shadow principal object of the admin forest: This script must be run in the admin forest with Domain Admin privileges of the forest root domain or Enterprise Admin privileges. Mandatory command line arguments:

–GoldCardUser 'the cn of the user to be added as member to the shadow group'
–ShadowGroup 'the cn of the shadow group'

Optional command line arguments:

–TTLMinutes 'minutes the membership should be valid' (default = 30)

Note: The given TTL must not be less than the max time skew for Kerberos authentication (default = 5 mins) or 0. Workflow:

get default msDs-ShadowPrincipalContainer from configuration naming context of admin forest
find shadow principal in default msDs-ShadowPrincipalContainer
build add request string depending on whether we passed a TTL time span or not
send add modification request for member attribute

More details in the code comments. [code lang="powershell"] #region params param ( # the cn of the user to be added as member to the shadow group [Parameter(Mandatory=$true)] [string]$GoldCardUser, # the cn of the shadow group [Parameter(Mandatory=$true)] [string]$ShadowGroup, # how many minutes should the membership be valid? # NOTE - if the remaining TTL is less than the max time skew for Kerberos auth # (default 5 mins) and not 0 -&amp;gt; no authentication will be possible [Parameter(Mandatory=$false)] [int]$TTLMinutes = 30 ) #endregion #region functions #region retrieve default msDs-ShadowprincipalContainer # from configuration naming context #endregion function GetShadowPrincipalContainer() { [string] $ret = "CN=Shadow Principal Configuration,CN=Services," # get a DC from current forest $dc = Get-ADDomainController -NextClosestSite -Discover # send UDP pipng to port 389 -&amp;gt; rootDSE call $rootDSE = Get-ADRootDSE -Server $dc.HostName.Value # read configurationNamingContext attribute $ret += "$($rootDSE.configurationNamingContext)" return $ret } #region get string to be used in LDAP add modification for member #endregion function BuildAddRequest([string]$goldCardCN, [int]$ttlMins) { [string] $ret = [String]::Empty # get goldcard admin for gold forest $goldmember = Get-ADUser -Filter { CN -eq $goldCardCN } # we have a TTL ? if ($ttlMins -ne 0) { $ttl = New-TimeSpan -Minutes $ttlMins # syntax must be &lt;TTL=TimeToLife in seconds,path to member&gt; $ret = "&lt;TTL=$($ttl.TotalSeconds),$($goldmember.DistinguishedName)&gt;" } else { # no TTL -&amp;gt; standard add modification call $ret = $goldmember.DistinguishedName } return $ret } #endregion #region main # get path to CN=Shadow Principal Configuration,CN=Services,configNC [string] $path = GetShadowPrincipalContainer # find gold shadow group in CN=Shadow Principal Configuration,CN=Services,configNC $goldshadow = Get-ADObject -Filter { CN -eq $ShadowGroup } -SearchBase $path -SearchScope Subtree # build value to ad to member attribute [string] $addreq = BuildAddRequest $GoldCardUser $TTLMinutes # add membership for goldcard admin with TTL Set-ADObject $goldshadow -Add @{ member = $addreq } # can only be used for group objects in a domain naming context: #Add-ADGroupMember 'testttl' -Members $goldmember -MemberTimeToLive (New-TimeSpan -Minutes $TTLMinutes) [/code] Update (10/31/2017): Some of us had an issue where production forest has child domains (not an uncommon scenario, isn't it?). Even though we added msDs-ShadowPricipals shadowing the Domain Admin SIDs of the child domains -> no logon / authentication with these SIDs to the child DCs was possible. Enterpise Admin SID shadowing didn't work either. Thus it looked like we only can use this security feature in the production forest root domain -> not OK. Mitigation:

logon to a DC in each child domain (if you have a multi-leveled forest tree, start from the top)
open 'Active Directory Domain and Trusts' management console
find incoming and outgoing parent<->child trusts with the parent -> open properties -> click
wait for replication to all DCs in the domain
restart all DCs
logon to a production forest root DC with an account from admin forest carrying either
- prodroot\Domain Admins and child\Domain Admins SIDs from msDs-ShadowPrincipal assignment
- prodroot\Entrprise Admins SID from msDs-ShadowPrincipal assignment
RDP to a child DC with this account >- should be successfull

Now you can RDP or authenticate with the shadowed SIDs from an admin forest machine directly to the child DCs. Update (9/29/2018) Added section How do the SIDs in the token come together: All the best and have fun PAMing. Michael PFE | Have keyboard. Will travel.

User provisioning and Password_Not_Required set - why?

Michael Frommhold MSFT — Fri, 17 Jul 2015 05:00:19 GMT

Today's topic:

User provisioning and Password_Not_Required set - why?

In large environments, it's definitely a clever idea having some Identity Management in place that - amongst other things - takes care for user provisioning. Many of the solutions out there - custom built or ready solutions - do utilize the Active Directory Service Interface (ADSI) or one of its wrappers (like System.DirectoryServices) to create user accounts. As a result we very often see the provisioned user accounts having the flag ADS_UF_PASSWD_NOTREQD (0x20) in userAccountControl attribute set. This is definitely a security risk since a domain admin can set a blank pwd on such an account which then can be used for logon. Question: Why did this happen? A common thinking error here is that you must first create the user and can only then set the password after the user is already in AD. Samples in the internet suggesting to create the user and then use the ADSI method SetPassword() after having the user created – like in the following code sample:

    DirectoryEntry user = null;

    try

    {

        // create user on dedicated dc to make sure we find the user 

        // for post processing (DCLocator vs. replication)

        DirectoryEntry ou = new DirectoryEntry("LDAP://" + dcName + "/" + ouPath,

                                                null, null, AuthenticationTypes.Sealing);

        // Add user to OU (note - we are still in the RAM - nothing happens in AD here)

        user = ou.Children.Add("CN=ADSI Test", "user");

        // set logon name (sAMAccountName)

        user.Properties["sAMAccountName"].Value = "adsitest";

        // set userPrincipalName

        user.Properties["userPrincipalName"].Value = "adsitest@mfp-labs.labsetup.org";

        // write from RAM to AD

        user.CommitChanges();

        Console.WriteLine("Success adding user");

    }

    catch (Exception ex)

    { Console.WriteLine("Adding user: {0} ({1})", ex.Message, ex.GetType()); }

This results in an account with - userAccountControl = 0x222 (ADS_UF_ACCOUNTDISABLE | ADS_UF_PASSWD_NOTREQD | ADS_UF_NORMAL_ACCOUNT) - pwdLastSet = 0 (never -> user must change password at next logon) Now we must set the initial password by invoking the method SetPassword of the underlying IADs object (can be inspected in user.NativeObject). SetPassword wraps the following tasks:

Bind to the Domain Controller we used in the creation task with AuthenticationType.Sealing set. Note – we need Kerberos enabled on the connection to be able to set the password.
Perform a replace modification request on the attribute unicodePwd against the path of the calling IADs object (NativeObject.AdsPath)

You see why we need to pre-create the user before we can use the SetPassword method?

    try

    {

        // this is a separate call into AD - no commit required

        // therefore we need the account to be present already

        user.Invoke("SetPassword", "P@ssWord");

        Console.WriteLine("Success setting pwd");

    }

    catch (Exception ex)

    { Console.WriteLine("Setting pwd: {0} ({1})", ex.Message, ex.GetType()); }

Now we have - userAccountControl = 0x222 (ADS_UF_ACCOUNTDISABLE | ADS_UF_PASSWD_NOTREQD | ADS_UF_NORMAL_ACCOUNT) - pwdLastSet = time of SetPassword() (user not must change password at next logon)

Saying – any domain admin may still reset the user’s password to a blank password. Answer: Because the following task very often just vanished in oblivion we stay with ADS_UF_PASSWD_NOTREQD flag set. We should must adjust userAccountControl and pwdLastSet to unset ADS_UF_PASSWD_NOTREQD flag and force the user to change password at next logon:

    try

    {

        // make account enabled / without pwd_not_req flag

        user.Properties["userAccountControl"].Value = 512;

        // force user must change pwd at next logon

        // we must pass a lon value -> cast 0 to long

        user.Properties["pwdLastSet"].Value = (long)0;

        // write from RAM to AD

        user.CommitChanges();

        Console.WriteLine("Success setting userAccountControl + pwdLastSet");

    }

    catch (Exception ex)

    { Console.WriteLine("Setting userAccountControl + pwdLastSet: {0} ({1})", ex.Message,

                                                                              ex.GetType()); }

Finally we have a provisioned account with acceptable settings. Question: Why is this a thinking error?

Answer: You guessed it - because we can set the password while creating a user account.

Question: But how – we have been reading above that you need to pre-create the user account before you can set the password? Answer: Just don’t use the method SetPassword in ADSI but perform the wrapped processing in your creation task. Saying - perform the replace modification on unicodePwd attribute while creating the user in RAM before writing into AD:

    static void SDS(string dcName, string ouPath)

    {

        DirectoryEntry user = null;

        try

        {

            // create user on dedicated dc to make sure we find the user 

            // for post processing (DCLocator vs. replication) and

            // set AuthenticationTypes.Sealing to enable pwd update

            DirectoryEntry ou = new DirectoryEntry("LDAP://" + dcName + "/" + ouPath,

                                                    null, null, AuthenticationTypes.Sealing);

            // Add user to OU (note - we are still in the RAM - nothing happens in AD here)

            user = ou.Children.Add("CN=ADSI Test", "user");

            // set logon name (sAMAccountName)

            user.Properties["sAMAccountName"].Value = "adsitest";

            // set userPrincipalName

            user.Properties["userPrincipalName"].Value = "adsitest@mfp-labs.labsetup.org";

            // pass pwd with account creation call

            user.Properties["unicodePwd"].Value = BuildBytePWD("P@ssW0rd");

            // make account enabled / without pwd_not_req flag

            user.Properties["userAccountControl"].Value = 512;

            // write from RAM to AD

            user.CommitChanges();

            Console.WriteLine("Success adding user");

        }

        catch (Exception ex)

        { Console.WriteLine("Adding user: {0} ({1})", ex.Message, ex.GetType()); }

    }

    private static byte[] BuildBytePWD(string pwd)

    {

        return (Encoding.Unicode.GetBytes(String.Format("\"{0}\"", pwd)));

    }

Alternatively we say good bye to ADSI and use the cool LDAP API wrapper System.DirectoryServices.Protocols:

    static void SDSP(string dcName, string ouPath)

    {

        // open LDAP connection to DC on port 389 (-> not GC => writable AD)

        LdapDirectoryIdentifier ldap_id = new LdapDirectoryIdentifier(dcName, 389, true, false);

        using (LdapConnection ldap_con = new LdapConnection(ldap_id))

        {

            // no kerberos -> no pwd set -> (Error = WILL_NOT_PERFORM)

            ldap_con.SessionOptions.Sealing = true;

            // bind to the DC

            ldap_con.Bind();

            // list of attributes we want to pass for creation

            List ldap_mod = new List { };

            // this has to be passed - otherwise we fail

            ldap_mod.Add(new DirectoryAttribute("objectClass", "user"));

            // set logon name (sAMAccountName)

            ldap_mod.Add(new DirectoryAttribute("sAMAccountName", "adsitest"));

            // set userPrincipalName

            ldap_mod.Add(new DirectoryAttribute("userPrincipalName", "adsitest@contoso.com"));

            // send the password with the creation request

            ldap_mod.Add(new DirectoryAttribute("unicodePwd", BuildBytePWD("P@ssW0rd")));

            // set userAccountControl to  Normal account

            ldap_mod.Add(new DirectoryAttribute("userAccountControl", (512).ToString()));

            // if we want to enforce pwd change on first logon - 

            // otherwise pwdLastSet is current time

            ldap_mod.Add(new DirectoryAttribute("pwdLastSet", (0).ToString()));

            // build add request

            AddRequest areq = new AddRequest("CN=ADSI test," + ouPath, ldap_mod.ToArray());

            try

            {

                // send reqeust to DC

                AddResponse aret = (AddResponse)ldap_con.SendRequest(areq);

                Console.WriteLine("Success");

            }

            catch (DirectoryOperationException doex)

            {

                Console.WriteLine("{0} ({1})", doex.Response.ErrorMessage,

                                               doex.Response.ResultCode);

            }

            catch (Exception ex)

            {

                Console.WriteLine("{0} ({1})", ex.Message, ex.GetType());

            }

        }

    }

    private static byte[] BuildBytePWD(string pwd)

    {

        return (Encoding.Unicode.GetBytes(String.Format("\"{0}\"", pwd)));

    }

OK - cool - now we have code avoiding provisioned user accounts with ADS_UF_PASSWD_NOTREQD flag set.

NOTE: When we try to set a password that does not meet complexity rules or minimum password length requirement of the domain, we lose a RID from the RID pool for every creation call. NTDS has to create the user internally to be able to write the given password to unicodePwd attribute -> we need a SID. If we fail with the password, the pre-created user object gets destroyed. The user object gets not deleted nor recycled - thus you will not be able to find info in the DB about this used RID. To avoid RID pool exhaustion, try to build proper password creation code and limit the retry loops in your IDM code. Additionally ask your beloved AD Admins to watch the RID pool on regularly bases.

Question: How do we get rid of this flag on already provisioned user accounts. We do not know whether a user account has a blank password or not and performing any sort of password hash dump has a big smell of hacking. Answer: We can rely on an Active Directory built-in security mechanism that takes care that no enabled user account with a blank password and ADS_UF_PASSWD_NOTREQD flag not set exists in AD.

When unsetting the ADS_UF_PASSWD_NOTREQD flag in userAccountControl on an enabled user account NTDS checks by comparing the user’s current password hash with the hash of a blank password. If we have a match NTDS will not perform the modification and throw an error stating that the current password does not meet complexity rules. Same will happen when enabling a disabled user account with a blank password and ADS_UF_PASSWD_NOTREQD flag not set.

Note – NTDS does not compare plaintext passwords – it simply doesn’t know these.

Therefore, a possible mitigation would be (thanks to Kevin Bell, PFE as well, for this idea):

find all user accounts (objectClass=user) that are of the type user and not computer (sAMAccountType=805306368) and have the ADS_UF_PASSWD_NOTREQD flag set in userAccountControl (userAccountControl:1.2.840.113556.1.4.804:=32)
walk the retrieved list of user accounts and unset ADS_UF_PASSWD_NOTREQD flag
if we fail with a password complexity error - report this user accounts as users with blank passwords

PoSh sample performing the above described tasks:

    $fishyusers = Get-ADUser -LDAPFilter "(&(objectClass=user)(sAMAccountType=805306368)" +

                                         "(userAccountControl:1.2.840.113556.1.4.804:=32))"

    foreach ($fish in $fishyusers)

    {

        try

        {

            Set-ADAccountControl -Identity $fish -PasswordNotRequired $false

            Write-Host "secured:" $fish.DistinguishedName

        }

        catch [Microsoft.ActiveDirectory.Management.ADPasswordComplexityException]

        {

            Write-Host "blank pwd for:" $fish.DistinguishedName

            // report into a log file?

        }

        catch

        {

            Write-Host $_.Exception.ToString()

        }

    }

What are the risks to do this in a production environment (?):

user accounts on which we could unset the flag successfully have no blank passwords and can go on working - but are more secure
user accounts on which we failed with password complexity error did not get modified - thus they can still work - but can be contacted and requested to set a complex password.
depending on the amount of user accounts that have been modified we may have increased replication traffic after the modifications. -> It could be a clever idea to do this out of work hours.

Hope you had some fun reading and wish fun with testing. History: 4/13/2016: added NOTE about possible excessive RID pool consumption. All the best Michael PFE | Have keyboard. Will travel.

Coding from the field

Resetting WMI repository - dos and dont's

Netlogon Debug helper

Automated Offline Domain Join using PowerShell and JEA

Abstract

Expected Result

Audience

Requirements

Technical documentation

PowerShell Restricted Endpoints - background

PowerShell Restricted Endpoints in this specific scenario

Security Aspects

The Deployment

Service Account

Create the endpoint in the internal network

Create the proxy endpoint in the internal network

Join a Machine Using the Automated Offline Domain Join

Appendix

RestrictedEndpoint LAN.ps1

RestrictedEndpoint DMZ.ps1

OfflineDomainJoinRequest.ps1

Deploy an ADLDS lab with replication and sync in less than an hour.

Abstract

Audience

Requirements

Expected Result

The Deployment

VMs and basic services

ADLDS Definitions

Requirements like DNS and Service Accounts

Installing ADLDS

Setting up LDAPS

Setting up ADAMSync

Visualize DSC Reporting with PowerBI

Abstract

Requirements

Desired Result

The heavy lifting behind the scenes

How to retrieve data from a JSON column

PowerBI - the interesting part

Starting with a new workbook

Using an existing workbook

Summary

Broken DSC Reporting, RequestEntityTooLarge and some LCM internals

Abtract

The change that leads to the issue

The effect on the database

Reporting works again

Some Richer Reporting

PowerShell Tip: Getting enum values as names, int and bit

Using SQL Server 2016 for a DSC Pull Server

Abstract

Credits

Requirements

Setting things ready on the SQL Server and connecting the pull server

Creating the DB on the SQL Server

Connecting the DSC Pull Server with the SQL database

The pull server must be configured to use MDB instead of the default EDB database

ODBC Data Source

Replacing the MDB with a version that is using "linked tables"

Testing the setup

Defining the configuration

Converting the configuration into a MOF

Publishing the MOF

Telling the DSC nodes to pull the TestConfig

Verifying the data on the SQL Server

Conclusion

Privileged Access Management - demystified

User provisioning and Password_Not_Required set - why?