Leveraging Windows Native Functionality to Capture Network Traces Remotely
Happy Monday everybody! Victor Zapata here again to share some network tracing and PowerShell love.
Like many of you, I’m frequently called on to troubleshoot issues that involve network connectivity of some sort. These issues can range from poor network performance to complete network failure… and almost always require collecting network traces on two or more computers simultaneously.
I still remember the days where collecting simultaneous network traces required installing network sniffer software on multiple computers along with the seemingly impossible task of coordinating various resources to start and stop the data collections at the same time. Oh boy… what a hassle!
Well we can happily put those sad days in our rear view mirror! Beginning in Windows 7/Windows Server 2008 R2, installation of network sniffer software is no longer necessary. Netsh functionality was extended in Windows 7/Windows Server 2008 R2 and is now capable of leveraging Event Tracing for Windows (ETW) to collect network trace data. If you’re not familiar with this functionality, check out Netsh Commands for Network Trace in Windows Server 2008 R2 and Windows 7 on TechNet.
For this implementation, we will be using the following two commands to start and stop network tracing on Windows Servers & Clients.
To start network tracing: NETSH TRACE START CAPTURE=YES TRACEFILE=<PATH>\<FILENAME.etl>
To stop network tracing: NETSH TRACE STOP
NOTE: There are some scenarios where network connectivity is dropped briefly when starting a network trace session using Netsh.
Things get even better beginning with Windows Server 2012 and later because PowerShell remoting is enabled by default. Because of this, we can use PowerShell to start, stop and collect network trace data programmatically and all from a central location without having to do anything special.
Here are the prerequisites to get this working:
1. Windows Server 2008 R2 / Windows 7 or later
2. PowerShell Remoting must be enabled (“winrm quickconfig”)
3. Administrator rights on all target computers
4. Network connectivity to each target computer (Firewall allows PS Remoting)
The following is an example of how to script the network trace data collection. To keep things simple, no error handling is included in this example.
DISCLAIMER: The sample code described herein is provided on an "as is" basis, without warranty of any kind.
#************************************************************************************#
# #
# This sample PowerShell script performs the following tasks #
# #
# 1. defines output path & folder on each target computer #
# 2. creates the output path & folder on each target computer if not already present #
# 3. starts network tracing on each target computer #
# 4. stops network tracing on each target computer when “x” key is pressed by user #
# 5. copies network traces from each remote computer to local directory #
# #
#************************************************************************************#
#START NETWORK TRACES ON REMOTE COMPUTERS
#Specify target computer names
$computers= "COMPUTER1","COMPUTER2"," COMPUTER3","COMPUTER4"
#Drive letter on remote computers to create output folder under
$drive="C"
#Folder path on remote computers to save output file to
$directory="TEMP\TRACES"
$path= $drive + ":\" + $directory
#SCRIPTBLOCK TO BE EXECUTED ON EACH TARGET COMPUTER
$scriptBlockContent=
{
param ($localoutputpath,$tracefullpath)
#Verify that output path & folder exists. If not, create it.
if((Test-Path -isValid $localoutputpath))
{
New-Item -Path $localoutputpath -ItemType directory
}
#Start network trace and output file to specified path
netsh trace start capture=yes tracefile=$tracefullpath
}
#Loop to execute scriptblock on all remote computers
ForEach ($computer in $computers)
{
$file= $computer + ".etl"
$output= $path + "\" + $file
Invoke-Command -ComputerName $computer -ScriptBlock $ScriptBlockContent -ArgumentList $path, $output
}
#Loop to check for “X” key
While($True)
{
$Continue= Read-Host "Press 'X' To Stop the Tracing"
If ($Continue.ToLower() -eq "x")
{
#STOP NETWORK TRACES ON REMOTE COMPUTERS
#Run 'netsh trace stop' on each target computer
ForEach ($computer in $computers)
{
Invoke-Command -ComputerName $computer -ScriptBlock {netsh trace stop}
}
#COLLECT TRACES
#Copy network traces from each target computer to a folder on the local server
ForEach ($computer in $computers)
{
$file= $computer + ".etl"
$unc= "\\" + $computer + "\" + $drive + "$\" + $directory
#Specify directory on local computer to copy all network traces to
#NOTE: There is no check to verify that folder exists.
$localdirectory="C:\TRACES"
$tracefile= $unc + "\" + $file
Copy-Item $tracefile $localdirectory
Write-Host $file "copied successfully to" $localdirectory
}
break
}
}
When the script runs, network traces are kicked off on all target computers using the computer’s name in the output file name. The tracing status should show as “Running” on each target.
NOTE: When collecting a series of traces (multiple traces in the same day) you may wish to ensure name uniqueness by adding something like the current time to the output file name. For example, the following will set the variable $TIMESTAMP with the current Month, Date, Year, Hours, Minutes and Seconds as a string value.
$timestamp = ((get-date).toString('MMddyyyyHHmmss'))
Then simply include $TIMESTAMP in the file name. ( Ex. $file= $computer + $timestamp + ".etl" )
Once the network tracing is started on all targets, simply reproduce the issue then press the “X” key followed by ENTER to stop tracing on all target computers.
Next, NETSH correlates the data collected and stops the trace sessions on all targets
Lastly, all network traces are copied to a local directory defined in the script
That’s it! Now the fun begins! Let’s open each one of these puppies and resolve this thing!
Victor Zapata