question

chetanVishwakarma-0048 avatar image
0 Votes"
chetanVishwakarma-0048 asked chetanVishwakarma-0048 rolled back

Compare local files with FTP Server files in Powershell

Hello Everyone ,

I am stuck in the middle of code , My code looks good to me only problem i am facing is that can't compare the file from my local computer to FTP server, its because the list of files i am getting in the variable is String without any columns header due to which i cant compare the file on FTP with local file,

Please help me to get this sorted out.

 Code: 
 $username='testUser'
 $password='Pwd'
 $ftp='ftp://abc.abc.org'
 $subfolder='/TestFolder'
 $ftpuri = $ftp + $subfolder
 $uri=[system.URI] $ftpuri
 $ftprequest=[system.net.ftpwebrequest]::Create($uri)
 $ftprequest.Credentials=New-Object System.Net.NetworkCredential($username,$password)
 $ftprequest.Method=[system.net.WebRequestMethods+ftp]::ListDirectoryDetails
 $response=$ftprequest.GetResponse()
 $strm=$response.GetResponseStream()
 $reader=New-Object System.IO.StreamReader($strm,'UTF-8')
 $list=$reader.ReadToEnd()
 $lines=$list.Split("`n")
 $lines

Output :,

PS C:\Users\vishwakarmac> $lines
06-03-13 10:30AM 17708 a a Call Tally Report FINAL.pdf
02-06-20 04:19PM <DIR> 2019
10-29-20 02:45PM 227288 2020_10_29_CassTM.pdf
12-17-20 04:24PM 226639 2020_12_17_Cass.pdf
02-25-21 08:06PM 226647 2021_02_25_Ct.pdf
08-04-21 01:21AM 226647 2021_04_22_Ct.pdf
07-19-21 11:30AM 226621 2021_06_24_Cat.pdf
08-26-21 03:44PM 227588 2021_08_2f.pdf
10-13-21 12:37PM 47192671 xyz.zip
03-08-18 03:40PM <DIR> Old


Variable details :
$lines.GetType()

IsPublic IsSerial Name BaseType


True True String[] System.Array



windows-server-powershell
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

cooldadtx avatar image
0 Votes"
cooldadtx answered cooldadtx commented

lines is a string array, not the FTP output. It should contain 1 element per file on the FTP server. Personally I wouldn't use ReadToEnd but rather enumerate line by line.

$reader = New-Object System.IO.StreamReader -ArgumentList $ftprequest.GetResponse().GetResponseStream()
while -not $reader.EndOfStream {
   $line = $reader.ReadLine()
   //Break up the line of data
}


You now need to break up each line to get the details of that line. I don't know what your FTP output looks like so you'll need to look at the output and figure out the actual parsing. I would daresay it is probably a series of fields separated by whitespace so for each line split on the whitespace again although it likely won't work for all cases. Here's a link to one possible approach. It is going to depend upon the output of your FTP server though. Ultimately you just end up breaking up the line into its fields and then storing that into a normalized structure you can then later use to do the comparison with the local files.

Looking at your example output you might be able to parse a line using just Split but limited to 4 entries: date, time, size and filename. By limiting to 4 the filename (with or without spaces) will be the final entry. If the size is <DIR> then it is a directory.


· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I am new to PowerShell I wanted to compare the zip file name and only date (10-13-21 12:37PM 47192671 xyz.zip) from ftp server to local zip file name and date , its because I will upload daily the same zip file to ftp and confirms whether the zip file successfully uploaded or not through zip file name and date.

Here is the logic you can understand:
abc.txt(Local folder)---> compress to xyz.zip ---> Upload xyz.zip to FTP server--> Confirm whether zip uploaded(Thas the reason i am checking filename and date with local filename and date ) and send email to us.

Hope you understood the whole process.

For SFTP i have created the script with Posh-SSH module but only for FTP i am facing issues.

I would appreciated if you can create powershell script according to the above logic Because i am SQL DBA and i know little bit PowerShell but not as expert as you.

Thanks in advance.

0 Votes 0 ·
cooldadtx avatar image cooldadtx chetanVishwakarma-0048 ·

Unless your tooling is broken, if you upload a file to FTP and you don't get an error back then it got there. I don't really see the need for then writing a script to verify it got there. That would be, to me, like copying files from one folder to another in Windows and then running a script to confirm all the files made it. The fact that the copy command didn't fail indicates the files got there successfully. The rest is just extra work for no real good reason.

Nevertheless, try something like this. I haven't fully tested it since I am not running the same FTP server you are.

0 Votes 0 ·
cooldadtx avatar image cooldadtx chetanVishwakarma-0048 ·
function Get-FtpFileDetails {
    param(
        [Parameter(Mandatory = $true)][string] $serverUrl,
        [PSCredential] $credential,
        [string] $path
    )

    $uri = $serverUrl    
    if (![String]::IsNullOrEmpty($path)) {
        $uri = [System.IO.Path]::Combine($uri, $path).Replace("\", "/")
    }

    $ftprequest = [System.Net.FtpWebRequest]::Create($uri)
    $ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::ListDirectoryDetails
    if ($credential) { $ftprequest.Credentials = $credential }
            
    $files = New-Object System.Collections.ArrayList
    $delimiters = @(' ', '`t')

    $reader = New-Object System.IO.StreamReader -ArgumentList $ftprequest.GetResponse().GetResponseStream()
    while (-not $reader.EndOfStream) {
        $line = $reader.ReadLine().Trim()
        if ($line.Length -eq 0) { continue }

        $tokens = $line.Split(" ", 4, [StringSplitOptions]::RemoveEmptyEntries)
        if ($tokens.Length -ne 4) { continue }

        $date = [System.DateTime]::ParseExact($tokens[0].Trim() + " " + $tokens[1].Trim(), "MM-dd-yy hh:mmtt", $null)    

        $size = 0
        if (-not [System.Int32]::TryParse($tokens[2].Trim(), [ref] $size) -or $size -eq 0) { continue }

        $filename = $tokens[3].Trim()
        $file = @{ "Filename" = $filename; "Size" = $size; "LastModifiedDate" = $date }
        $files.Add($file)
    }

    $reader.Close()    

    return $files
}
0 Votes 0 ·
RichMatheisen-8856 avatar image
0 Votes"
RichMatheisen-8856 answered

See if this is helpful:

 $raw = @"
 06-03-13 10:30AM 17708 a a Call Tally Report FINAL.pdf
 02-06-20 04:19PM <DIR> 2019
 10-29-20 02:45PM 227288 2020_10_29_CassTM.pdf
 12-17-20 04:24PM 226639 2020_12_17_Cass.pdf
 02-25-21 08:06PM 226647 2021_02_25_Ct.pdf
 08-04-21 01:21AM 226647 2021_04_22_Ct.pdf
 07-19-21 11:30AM 226621 2021_06_24_Cat.pdf
 08-26-21 03:44PM 227588 2021_08_2f.pdf
 10-13-21 12:37PM 47192671 xyz.zip
 03-08-18 03:40PM <DIR> Old
 "@
    
 $file = "C:\Junk\upn.csv"
 $leaf = Split-Path $file -Leaf
 $lastwrite = Get-Date (get-item $file).LastWriteTime
    
 $lines = $raw -split "`n"
 ForEach ($line in $lines){
     $date,$time,$size,$name = $_.split(" ")
     if ($size -ne '<DIR>' -and $name -eq $leaf){
         # if clocks on both machines aren't closely synchronized
         # this might need some work
         # if time zones on both machines are different 
         # this might also need some work
         # if you're only interested in the date and the time is
         # immaterial then just get the DATE and ignore the time
         # but that may still be a problem if the creation/upload
         # time spans the midnight boundary
         # Doing "time stuff" always presents problems!
         if ( (Get-Date ("$date $time")) -ge $lastwrite ){
             Write-Host "Upload OK"
         }
         else{
             Write-Host "FTP file older than $file"
         }
         break   # no need to check any other files
     }
 }
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

chetanVishwakarma-0048 avatar image
0 Votes"
chetanVishwakarma-0048 answered chetanVishwakarma-0048 rolled back

Thanks for sharing the code I took help from your code and managed to develop the code with my understanding , please let me know if any issues can be there with the code.

 $username='username'
     $password='password'
     $ftp='ftp://ftp.xyz.org'
     $subfolder='/folderS'
     $ftpuri = $ftp + $subfolder
     $uri=[system.URI] $ftpuri
     $ftprequest=[system.net.ftpwebrequest]::Create($uri)
     $ftprequest.Credentials=New-Object System.Net.NetworkCredential($username,$password)
     $ftprequest.Method=[system.net.WebRequestMethods+ftp]::ListDirectoryDetails
     $response=$ftprequest.GetResponse()
     $strm=$response.GetResponseStream()
     $reader=New-Object System.IO.StreamReader($strm,'UTF-8')
     $list=$reader.ReadToEnd()
     $lines=$list.Split("`n")
        
     $count = $lines.Count
     $dateToMatch = Get-Date -Format "MM-dd-yy"
        
     Function left {
        [CmdletBinding()]
         
        Param (
           [Parameter(Position=0, Mandatory=$True,HelpMessage="Enter a string of text")]
           [String]$text,
           [Parameter(Mandatory=$True)]
           [Int]$Length
        )
     $left = $text.SubString(0, [math]::min($Length,$text.length))
     $left
     }
     For ($i=0; $i -le $count; $i++) 
     {
            
         $var = $lines[$i]
         if($var -like "*Brokers.zip*")
         {
             $date =  left -text $var -Length 8
             if($date -eq $dateToMatch)
             {
                 Write-Host "Matched"
                     $date
                     $ftpFile
             }
             else 
             {
                 Write-Host "No file uploaded on FTP"
             }
         $ftpFile = "Brokers.zip"
           
         }
         
            
     }

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.