Invoke-command -computername (get-content \\utilityserver\serverlist\servers.txt) -scriptblock {import-module webadministration;Restart-WebAppPool -name myapppool}

Or for all the apppools

Invoke-command -computername (get-content \\utilityserver\serverlist\servers.txt) -scriptblock {import-module webadministration;gci 'IIS:\AppPools' -verbose | Restart-WebAppPool}

But suddenly I found out that it was not as easy as I thought as PowerShell remoting was not set up , I had to look at other alternatives like a for loop in conjunction with psexec and appcmd to get the job done. This greatly bugged me though, I remembered that IIS had changed the WMI namespace to root\webadministration from root\MicrosoftIISv2, and should be able to use PowerShell to remote in through WMI and recycle the apppools without PowerShell remoting. Researching IIS WMI and powershell I found that there was not much documentation on it that can be really helpful. In the process of writing this it turned into a PowerShell advanced function that will

• Recycle all apppools on remote server
• Recycle specified apppool on remote server
• Take pipeline inputs of server names
• Check for application pool existence before executing the recycle
• Check if server is online before executing the Script.

Here is the advanced function, load this into your profile and use it like a cmdlet

PS C:\Windows\system32> get-help Operationsguy-Recycleapppool

Operationsguy-Recycleapppool [-Server] <String> [-apppool] <String> [-Verbose] [-Debug] [-ErrorAction <ActionPreference>] [-WarningAction <ActionPreference>] [-ErrorVariable <String>] [-WarningVariable <String>] [-OutVariable <String>] [-OutBuffer <Int32>]

Function Operationsguy-Recycleapppool{
[CmdletBinding()]
param(
[parameter(Mandatory=$true, Position=0,ValueFromPipeLine= $true)]
[string]$Server,
[parameter(Mandatory=$true,Position=1)]
[string]$apppool
)
process{
if ((Test-Connection $Server -count 1 -ErrorAction SilentlyContinue) -and ($apppoolarray=gwmi -namespace "root\webadministration" -Class applicationpool -ComputerName $Server -Authentication 6 -Property name -ea 'SilentlyContinue' | select -ExpandProperty name))
{
if ($apppool -eq "ALL")
{gwmi -namespace "root\webadministration" -Class applicationpool -ComputerName $Server -Authentication 6 | Invoke-WmiMethod -Name recycle -ErrorAction SilentlyContinue
if ($?)
{Write-Host "All apppools recycled on $server"}
else {Write-Host -BackgroundColor Red "One apppool is either stopped or did not start backup"}
}
elseif ($apppool)
{if ($apppoolarray -contains $apppool)
{
gwmi -namespace "root\webadministration" -ComputerName $server -Authentication 6 -Query "select * from applicationpool where name='$apppool'" | Invoke-WmiMethod -Name recycle -ErrorAction SilentlyContinue
if ($?)
{Write-Host "$apppool recycled on $server"}
else
{Write-Host "$apppool Apppool state Error"}
}
else
{Write-Host -BackgroundColor Red "$apppool does not exist on $server"}}
}
else
{Write-Host -BackgroundColor Red "$server not reachable or WMI Windows feature for IIS not installed"}
}
}

e.g: "localhost","mypc" | Operationsguy-Recycleapppool -apppool all

get-content d:\servers\server.txt | Operationsguy-Recycleapppool -apppool myapppool

For this to work WMI connectivity between management server and remote hosts is needed, also IIS Management Scripts and Tools component under Management Tools needs to be installed on the remote IIS 7 or 7.5 machines. This will not work for IIS 6.0 servers (try replacing the namespace with root\MicrosoftIISv2 in the function, I have not tried it it may work).

See you guys soon with another post on AD PowerShell cmdlets and how it can be used to audit your groups and get some good information about your servers.

for /f %i in (\\utilityserver\servers.txt) do schtasks /s %i /create /TN custom_task /TR "powershell -nologo -file c:\localbin\task.ps1" /ST 16:00 /SC MINUTE /MO 5 /RU <Domain\user> /RP "XXXXX"

The tasks were created fine on all the machines but when I tried to run , it failed. Tried executing the powershell script locally on a server and it threw me a error message about execution policy. Now I have to enable the execution policy on around 100 servers which unfourtunately did not have powershell remoting setup. When you set a execution policy in powershell it actually modifies registry value for ExecutionPolicy at the following location.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\ShellIds\Microsoft.PowerShell ( I found this by running procmon) .

If you have a unrestricted policy your registry will read like this

reg query  HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\ShellIds\Microsoft.PowerShell

    Path    REG_SZ    C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
    ExecutionPolicy    REG_SZ    Unrestricted

Now to set this across 100 machines

for /f %i in (\\utilityserver\servers.txt) do reg add \\%i\HKLM\SOFTWARE\Microsoft\PowerShell\1\ShellIds\Microsoft.PowerShell /v ExecutionPolicy /t REG_SZ /d Unrestricted /f

Replace the value with Unrestricted | RemoteSigned | AllSigned | Restricted | Bypass which ever you want to set. This key will set the execution policy for all the users on a machine. You can also use the set-execution policy cmdlet if you have powershell remoting setup.

This will save you a bunch of time , or I will suggest you make this a part of your build process.

Error activating the worker process manager instance within the worker process. - Inner Exception: System.InvalidOperationException: Unable to activate worker process proxy object within the worker process: ipc://8ce29399-c3b3-4c1f-aec8-2e1bf132b0dd:7000     at Microsoft.SharePoint.UserCode.SPUserCodeWorkerProcess.CreateWorkerProcessProxies

Trying to find out if it was really connecting to the web proxy, I fired up netmon that is when I found the interesting trace below, even though the proxy node is set up in the config files the sandboxed solution is not honoring it, it is directly trying to connect and retransmitting again and again till we reach the maximum TCP retransmits

 Now I am sure what the cause of the issue is but still not sure about the solution, turned to my fav search engine (www.bing.com) J .Reading multiple sources on how applications can use proxies I reached at the following conclusion any .net applications on IIS will use .net proxy in the web.config, but there is also a case where some applications can use Winhttp settings for proxies.  Having ruled out the .net proxy, I decided to set up some winhttptracing to see if I can find anything. Here is an excellent resource on how to do it https://blogs.msdn.com/b/jpsanders/archive/2009/05/28/how-to-enable-winhttp-tracing-on-vista-2008-and-windows-7.aspx.  What I am expecting at this point to see any signs of the external URL call in the winhttptrace, you can use netmon to read through an etl trace slick isn't it? Looking at the WINHTTP_MicrosoftWindowsWinHttp conversation that has a WEBIO_MicrosoftWindowsWebIO branch I could see

.

 Now convinced that the sandboxed code solution is using Winhttp proxy to make outbound URL calls I set the winhttp using netsh referring https://technet.microsoft.com/en-us/library/cc731131(WS.10).aspx

netsh winhttp proxy>set proxy-server="http=<internal proxy>:80" bypass-list= "*.<machine domain>.com"

PS: Make sure that you add the bypass list for all the internaldomains that needs to bypassproxy otherwise you will have a situation where your backend communication will go through proxy which you may not want. 

After setting the winhttpproxy the sandboxcodesolutions was working as expected. One more issue resolved many to come. 

·     Export the event log with a scheduled task

·     Write the exported log to a remote share

·     Time stamp the exported log

·     Provide access to the share for the development team 

When operation engineers think about scheduled tasks the first thing they think is the overhead of managing a account and changing passwords on that , since windows 2008 we have the capability of running scheduled tasks under network service which is great as it takes away the overhead of password management. One point to be noted is if you are running a process under network service that needs access on network resources they are accessed under the identity of machine account. You can export the event log by using the windows 2008 and higher by the command line utility wevtutil and date, time stamp the file using PowerShell cmdlet get-date . Assume you write to the remote share \\utilityserver\logdump  and the machine account where you are exporting the logs via scheduled tasks has write access to the share.

On the utility server create shares and provide necessary permissions using the following

net share logdump=D:\eventlogdump /grant:<domain\machineaccount>$,change

icacls D:\eventlogdump  /GRANT <domain\machineaccount>:(D,WDAC)

On the machine exporting logs save the following as logexport.ps1

$share="logdump"
$utilityserver="noname"
wevtutil epl application \\$utilityserver\$share\$(get-date-uformat "%Y_%m_%d_%H_%M_application_$(hostname)").evtx

Run the following to create a scheduled task on the machine

schtasks /create /TN log_export /SC MINUTE /MO 10 /TR "powershell -file c:\temp\logexport.ps1" /RU "networkservice"

Once the job kicks off you can see a file similar to 2011_01_05_14_46_application_web1.evtx every 10 minutes.   If you want to deploy to number of machines copy the logexport.ps1 to the machine and use

schtasks /s <remotemachinename> /create /TN log_export /SC MINUTE /MO 10 /TR "powershell -file c:\temp\logexport.ps1" /RU "networkservice"

Alternatively if you do not want to retry the command all the time export the scheduled task as an xml file and re-import the xml file on any number of machines.

For shared hosting environments where you need segregation between application errors the best way is to log parse the eventlogs, filter it and insert it into a Database and build a UI that enforces role based access control for different applications. The application owners can access the UI to see the exceptions in production. This is how error logs are passed on to development teams in the shared hosting model of www.microsoft.com we will cover that in a different blog post.

In the next post we will look at a scenario where I had fun with Sandbox code solutions in SP2010.

I started looking at it . That is when I understood there is a reason why security thinks it  not "secure". I dumped the winrm default config on my local machine running windows 2008 r2 

C:\Users\vbaby>winrm enumerate winrm/config/Listener
Listener
    Address = * -- All the configured addresses.
    Transport = HTTP
    Port = 5985   -- Default port
    Hostname
    Enabled = true
    URLPrefix = wsman
    CertificateThumbprint
    ListeningOn = 192.168.0.4, 127.0.0.1, ::1 --- All IPs are listening for Winrm

Now  I am thinking the same as my friend we need to change this in prod environment.  All the machines in this scenario has 2 interfaces Customer facing, Admin. The security suggested measure was to lock WINRM on the admin interface on a non standard port and get the port opened through the Back end firewall.  I could come up with a script  in some time to do this as follows

gci WSMan:\localhost\Listener | Remove-Item -Recurse 
$port="23001"
$IP=gwmi Win32_NetworkAdapterConfiguration | ? {$_.ipaddress -like '198*'} | select -expandproperty ipaddress
new-item -path WSMan:\localhost\Listener -Transport http -Address "ip:$IP" -Port 23001 -force

The first line removes the default listener, second line sets a non standard port, third line find the interface here all the admin interfaces started with 198. Fourth line created a listener  on the IP at the specified port.

Asked him to save it as <name>.ps1 and run it on all the machines he want to configure.  Send this off to him . That evening I receive an extension or the same request  " I need to run this on 200 + machines help me run it remotely".  

Looking at it I decided to use Psexec . Testing it Psexec was looking okay , only problem was I had to press enter twice after each execution , this was not looking good suddenly this is turning complex . Doing some online searches I found this https://www.leeholmes.com/blog/2007/10/02/using-powershell-and-psexec-to-invoke-expressions-on-remote-computers/ . Things were looking good after that . I saved the above script as remoting.ps1 on to  my utility server after that used a for loop to loop through my servers to test it. Here is the for loop

for /f %i in (\\utilityserver\servers\server.txt) do psexec \\%i /u domain\<user> /p <pass> cmd /c "echo .| powershell -file \\utilitserver\scripts\remoting.ps1"

PsExec v1.96 - Execute processes remotely
Copyright (C) 2001-2009 Mark Russinovich
Sysinternals - www.sysinternals.com

   WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Listener

Name                      Type                 Keys
----                      ----                 ----
Listener_1226527951       Container            {Address=IP:192.168.0.10, Tran...

cmd exited on web9  with error code 0.

Now I have new listener on all may machines exactly the way I wanted on a particular port , on a particular IP all set up remotely . The only issue  is when I run remote power shell as invoke-command tries to 5985 by default.

 Invoke-Command -ComputerName web19 -ScriptBlock {gci cert:\localmachine\my}

[web19] Connecting to remote server failed with the following error message : The client cannot connect to the destination specified in the request. Veri
    + CategoryInfo          : OpenError: (:) [], PSRemotingTransportException
    + FullyQualifiedErrorId : PSSessionStateBroken

I have to specify the port now

Invoke-Command -ComputerName web19  -ScriptBlock {gci c:\ } -Port 23001

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----          4/8/2010  12:08 PM            compaq
d----          4/8/2010  12:10 PM            CPQSYSTEM
d----          4/8/2010  12:08 PM            hp
d----          7/2/2010   7:57 AM            inetpub
d----          7/2/2010   8:00 AM            localbin
d----         7/13/2009   8:20 PM            PerfLogs
d-r--          8/8/2010  11:19 AM            Program Files
d-r--          8/8/2010  11:19 AM            Program Files (x86)

Looks like problem solved. Send this off to my friend , got  back a reply all good.

If you are interested to know how WSMAN works over a port when you even do not have IIS on a machine , try this from command prompt

netsh http show urlacl

Reserved URL            : https://+:23001/wsman/
   User: NT SERVICE\WinRM
       Listen: Yes
       Delegate: No
   User: NT SERVICE\Wecsvc
       Listen: Yes
       Delegate: No
       SDDL: D:(A;;GX;;;....)

You can see that the listener is created at the http.sys level.

In the next post we will look at having some fun with scheduled tasks and wevtutil.

• Command line - Using Winhttpcertcfg
• Powershell - Just to explore the new cooler side

 Winhttpcertcfg was primarily for windows 2000/2003 but it works well on windows 2008 & r2 :). You can download it at https://www.microsoft.com/downloads/en/details.aspx?familyid=c42e27ac-3409-40e9-8667-c748e422833f&displaylang=en  and install it or extract contents out of it you only need winhttpcertcfg.exe. Copy it to your utility server . As always I am using server names out of a text file, also using psexec as winhttpcertcfg does not have remoting capabilities.( If  you do not want to do psexec you can use the Powershell way shown in the next section)

for /f %i in (servers.txt) do psexec \\%i  /U domain\uname /P "XXX" \\utilityserver\tools\winhttpcertcfg\winhttpcertcfg.exe -g -c LOCAL_MACHINE\MY -s "certsubjectname" -a "\network service"

where network service is the  name of the application pool identity

You can also install the cert if you have the pfx and give access to the apppool identity in  a single step using the below

for /f %i in (servers.txt) do psexec \\%i /U domain\uname /P "XXX" \\utilityserver\tools\winhttpcertcfg\winhttpcertcfg.exe -i \\certserver\certs\foo.pfx  -c LOCAL_MACHINE\MY  -a "\network service"

Let us do the same using Powershell, it turned out to be a bit more complex following is a script I wrote that takes in a thumbprint and apppool identity that needs access as the parameter.

(Disclaimer : Please use this at your own risk , test once , twice , thrice in your test environment to make sure it produces intended results)

if ($args[0] -eq $null -or $args[1] -eq $null){

      Write-Host "Insufficient parameters"
      Write-Host "Usage:certaccess.ps1 thumbprint username"

      exit}

else {$TP=$args[0]
$uname=$args[1]
$keyname=(((gci cert:\LocalMachine\my | ? {$_.thumbprint -like $tp}).PrivateKey).CspKeyContainerInfo).UniqueKeyContainerName
$keypath = $env:ProgramData + "\Microsoft\Crypto\RSA\MachineKeys\"
$fullpath=$keypath+$keyname
icacls $fullpath /grant $uname`:RX
}

Save the above script as certaccessgrant.ps1. Make sure all the machines you are going to run the script is set up for powershell remoting and has the cert installed. After that

invoke-command -computername (get-content servers.txt) -filepath c:\temp\certaccessgrant.ps1 -argumentlist ‎118630e5ce55a52dwhlklklSWEwe42qbac2bb4b47bf, testuser

where 18630e5ce55a52dwhlklklSWEwe42qbac2bb4b47b is the thumbprint and testuser is the user name that is apppool identity that needs access.

Also if you are dealing with 1 or 2 machines (Windows 2008 & up)  you can go the slowest GUI way of  start --> run -->MMC--> certificates --> local machine --> cert store

Right click the cert , manage private keys and provide access  to the apppool identity on all the machines. In windows 2003 the GUI way is using this tool wsecertificate2.exe details here at https://msdn.microsoft.com/en-us/library/ms824698.aspx.

if you are looking for a download location https://www.microsoft.com/downloads/en/details.aspx?FamilyID=56fc92ee-a71a-4c73-b628-ade629c89499&displaylang=en

The best features of tinyget is

• You can control the headers that is passed (very handy if you want to do some specific user agent testing )
• You can check the status against an expected result .
• You can do Looping for requests.
• You can test the content body for strings.
• Test web pages that use SSL
• Test web pages that use authentication or client certificates.

the list goes on

Continuing from the earlier post we are going to look at how to test for Content-length if a server hosts multiple websites . Here we use a for loop  to execute against a set of servers from temp.txt against www.site1.com on a server that hosts different sites

for /f %i in (C:\temp\server.txt) do @echo %i && @c:\tools\tinyget.exe -srv:%i -status:200 -uri:"/microsoft/foo.aspx" -rh:"Host: www.site1.com\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 7.0;Windows NT 5.1)\r\n" -h | findstr "Content-Length:"

web33
Content-Length: 60
web34
Content-Length: 60
web35
Content-Length: 60
web36
Content-Length: 60
web37
Content-Length: 60
web38
Content-Length: 60
web39
Content-Length: 22 (Broken)

 Let us look at another example from the previous Part 1 post where one of you machines is serving a 404  and how to capture it with tiny get . it is easy , using the same example , you look for a status of 200 ok to single out the machine that is throwing 404

for /f %i in (C:\temp\server.txt) do @echo %i && @c:\tools\tinyget.exe -srv:%i -status:200 -uri:"/microsoft/foo.aspx" -rh:"Host: www.site1.com\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 7.0;Windows NT 5.1)\r\n"

web33
web34
web35
web36
web37
ERROR: 0x1 : Testcase number: 0 - Explain: (null)
ERROR: 0x4b8 : returned status code (404) does not match expected one (200)
ERROR: 0x1 : URI: /microsoft/foo.aspx, SSL: Nonsecure, CliCert:(null), Auth:Anon Domain:(null) User: (null) Password: (null)Received status/error info: 404 Not Found - HTTP 404. The resource you are looking for (or one of its dependencies) could have been removed, had

If you are looking at looping a particular url against a a set of machines you can use same command with the highlighted options.

for /f %i in (C:\temp\server.txt) do @echo %i && @c:\tools\tinyget.exe -srv:%i -status:200 -uri:"/microsoft/foo.aspx" -rh:"Host: www.site1.com\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 7.0;Windows NT 5.1)\r\n" -x:10  -l:10

 The above will run 10 threads each 10 times against each server for the url :"/microsoft/foo.aspx". This is not advised to stress test for stress testing you shoud use WCAT from the same resource kit or download from www.iis.net , but this  is great in case you want debug some perf  issues for a particular url .  

 Recently was dealing with a case where there was a typo on a page that was published out , suddenly the content publisher noticed it and reverted the changes . But while testing it he was seeing intermittently the page that had typo . This is what I did to isolate the issues to proxy cache ie, first prove none of my machines was serving the page  that had typo .

for /f %i in (C:\temp\server.txt) do @echo %i && @c:\tools\tinyget.exe -srv:%i -status:200 -uri:"/microsoft/foo.aspx"  -testcontainstring:"<correct string>" -rh:"Host: www.site1.com\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 7.0;Windows NT 5.1)\r\n"

if one of my servers was broken the below would have been my output
web37
ERROR: 0x4b8 : response body does not contain expected string
Expected:<correct string>
Received: <spew of the page content>

Finally if you are testing against a url that is  secure (https)  use the highligted option

for /f %i in (C:\temp\server.txt) do @echo %i && @c:\tools\tinyget.exe -srv:%i -status:200 -uri:"/microsoft/foo.aspx" -s:3-rh:"Host: www.site1.com\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 7.0;Windows NT 5.1)\r\n"

This concludes the 2 part series. In the next post we will cover some fun with permission for cert private keys.

1) Launch power shell from a windows 2008 ( with IIS7 powershell module installed ) or windows 2008 r2 machine

Add-pssnapin WebAdministration (Windows 2008)

or

import-module webadministration (Windows 2008 r2 )

2) Get the server names from a text file and pipe it to the get-weburl cmdlet .

To look at content lengh if the issue is about out dated content showing up intermittently on server farm

Get-Content c:\temp\servers.txt  | % {$_;get-weburl -Url https://$_/my/sites/foo.aspx -ResponseHeaders| %{($_.headers).'Content-Length'}}

WEB01
20134
WEB02
20134
WEB03
20134
WEB04
20134
WEB05
20134
WEB06
20134
WEB07
20051 (Broken)

To find a machine which throws 404 or 500 in  cluster .

Get-Content c:\temp\servers.txt  | % {$_;get-weburl -Url https://$_/my/sites/foo.aspx -ResponseHeaders| select responseuri, staus, description

ResponseUri                                  Status                                  Description
-----------                                                   ------                                  -----------
https://WEB01/my/sites/foo.aspx ProtocolError                           URL Rewrite Module Error. (Broken)
https://WEB02/my/sites/foo.aspx OK                                      OK
https://WEB05/my/sites/foo.aspx OK                                      OK
https://WEB06/my/sites/foo.aspx OK                                      OK
https://WEB07/my/sites/foo.aspx OK                                      OK
https://WEB08/my/sites/foo.aspx OK                                      OK
https://WEB21/my/sites/foo.aspx OK                                      OK
https://WEB22/my/sites/foo.aspx OK                                      OK
https://WEB23/my/sites/foo.aspx OK                                      OK
https://WEB24/my/sites/foo.aspx OK                                      OK
https://WEB25/my/sites/foo.aspx OK                                      OK
https://WEB26/my/sites/foo.aspx OK                                      OK
https://WEB27/my/sites/foo.aspx OK                                      OK
https://WEB28/my/sites/foo.aspx OK                                      OK
https://WEB41/my/sites/foo.aspx OK                                      OK
https://WEB42/my/sites/foo.aspx OK                                      OK
https://WEB43/my/sites/foo.aspx OK                                      OK

OK is a 200.

Well this works great if your server has one website , what if you are doing shared hosting business with multiple sites on the same server,  that is when we travel back in time  to meet IIS 6 resource kit tool tinyget. That will be covered  in the next post.

1) Change the password in AD for service account.

2) Update the password for the service accounts in the sharepoint 2010 farm .

Here is a quick run through

Change the password in AD for service account.

I assume  you can log on to a windows 2008 r2 server in domain where service account exists , you have rights to change the password for the service account. I am using a fictitious account  by the name _svc_acct for this scenario

a) After logging into a server in account domain  launch powershell

b ) Load the AD module

import-module activedirectory

c) Assign the account name to a variable

     $account="_svc_acct"

d) set the password ( please note this as we will be using this in the next section)

     Set-ADAccountPassword -Identity $account -OldPassword (ConvertTo-SecureString -AsPlainText  "xxxx" -Force) -NewPassword (ConvertTo-SecureString -AsPlainText "xxxxx" -Force)

e) Check for successful update of the password updation  ( out put should show recent tiem stamp)

Get-ADUser $account -properties * | select PasswordLastSet

For engineers who do not have windows 2008 r2 machines , you can always change password through your normal process.

Update the password for the service accounts in the sharepoint 2010 farm .

 a) Log on to the CA in the Farm, launch powershell.  Assign the new password from step d in the above section to a variable also store the account to a variable

$account="_svc_acct"

$securepassword=convertto-securestring "xxxxx" -asplaintext -force

b) Retrieve the Sp managed account   and pipeline it to the set-spmangedaccount cmdlet

Get-SPManagedAccount Domain\$account | Set-SPManagedAccount -ExistingPassword $securepassword -UseExistingPassword -confirm:$false

Guys tell me this is easy !!