How to check the new entries of a file that might contain a specific error

Question

I have a log file and I am trying not to capture the same error in the log when the below script runs. The script is running every 5 minutes through task scheduler. (so, if there are new entries with the specific error an email should be sent, if not then nothing to do - it shouldn't read the whole log) The goal is to store the last time the log was checked with a time stamp. If there is no new entry (of the same error) when the script runs then there is nothing to do. (shouldn't send an Email) Else it should send an Email.

The log looks like:
2021-01-05 15:44:45
CODE : 00002
LEVEL : 4
NAME : com.apps.common.util.ApplicationException
ERROR # : UTUD3SIQY3Z0
TOKEN ID : 2574333
USER : System
MESSAGE : The requested operation could not be completed.
at com..apps.common.util.LoggingHelper.logError(LoggingHelper.java:128)
at com..apps.common.util.ErrorHandler.log(ErrorHandler.java:644)
at com..apps.common.util.ErrorHandler.handleError(ErrorHandler.java:131)

What I have tried so far but without success is this:

$File = 'D:......\logs\Nikos_text2.txt'
$content = Get-Content $File
While($true){
$date = (Get-Date).AddMinutes(-5)
$Writetime = Get-Item $File | select lastwritetime | get-date
If ($Writetime -le $date -and $content -like 'Trigger action error'){
$smtp = new-object Net.Mail.SmtpClient("server name")
$objMailMessage = New-Object System.Net.Mail.MailMessage
$objMailMessage.From = "support@keyman .com"
$objMailMessage.To.Add("support@keyman .com")
$objMailMessage.To.Add("support@keyman .com")
$objMailMessage.Subject = "Error message: 'Trigger action error'"
$objMailMessage.Body = "In log in server (SI) I traced the following error message: Trigger action error. Pls check it asap"
$smtp.send($objMailMessage)

}
Start-Sleep -Seconds 300
}

From my understanding either I should capture every time the script runs the last line of the log that previously run or put a timestamp and the next time it will run, then to check for errors after the last timestamp. But due to the fact that I am brand new to Powershell, I don't have a clue how to achieve it.
Any help would be much appreciated.

Thank you in advance

I now made some changes to my code:

$time = '{0:yyyy-MM-dd HH:mm:ss}' -f (Get-Date '2021-02-02 12:40:48').AddMinutes(-5)
$File = 'D:......\logs...log'
$search = 'Application Error*'
$query = (Get-Content $File) | Where-Object {$-match $search} | Select-Object -Last 1
if ($query -match "Application Error*" -and ($ -split ',')[0] -gt $time) {
$smtp = new-object Net.Mail.SmtpClient("server")
$objMailMessage = New-Object System.Net.Mail.MailMessage
$objMailMessage.From = "support@keyman .com"
#$objMailMessage.To.Add("support@keyman .com")
$objMailMessage.To.Add("support@keyman .com")
$objMailMessage.Subject = "Error message: 'Application error'"
$objMailMessage.Body = "In log in server I traced the following error message: Application error"
$smtp.send($objMailMessage)
}

but still I am not able to receive an Email notification.

Answer 1

See if something like this works for you:

$fn = "C:\junk\1.txt"
$cycletime = 5  # in minutes
$lastcheck = (get-date).AddMinutes(-$cycletime)   # initialize the time
While ($true){
    $writetime = Get-ItemProperty  $fn | Select-Object LastWriteTime | Get-Date
    if ($writetime -ge $lastcheck){     # no need to check the log if it hasn't been written to
        # get all possible matches and select only the last (most recent) one
        $all = Select-String -Path $fn -Pattern "^NAME\s:\s.+\.ApplicationException$" -Context 3,4 | 
            Select-Object -Last 1
        # get the date from the error
        $lastspecificerrortime = Get-Date $all.Context.PreContext[0]
        # if the error happened since the last check, report it
        if ($lastspecificerrortime -gt $lastcheck){
            # send your e-mail here
        }
    }
    $lastcheck = Get-Date
    Start-Sleep -Seconds ($cycletime * 60)
}

I didn't check to see if there were no errors selected, so you probably should do that!

Answer 2

Give this a try:

$fn = "C:\junk\1.txt"
$cycletime = 5  # in minutes
$linestocheck = 500             # only check the last X number of lines in $fn
$startcheck = 0
$lastcheck = (Get-Date).AddMinutes(-$cycletime)   # initialize the time
$regex = "\<Error\>\<(.+?\s.+?)\>\s.+Content is not allowed in prolog\.$"  # store the regex
While ($true) {
    $startcheck = Get-Date                          
    $writetime = Get-ItemProperty  $fn | Select-Object LastWriteTime | Get-Date
    if ($writetime -ge $lastcheck) {
        # no need to check the log if it hasn't been written to
        $latestmatch = (Get-Date).AddMinutes(-15)               # give (more than) enough overlap when looking for errors 
                                                                # maybe change to AddMinutes(-(3*$cycletime))???
        Get-Content -Path $fn -Tail $linestocheck |
            ForEach-Object {
                if ($_ -match $regex) {
                    $errordate = [datetime]$Matches[1]
                    if ($errordate -gt $latestmatch) {
                        $latestmatch = $errordate
                    }
                }
            }
        # if the most recent error happened since the last check, report it
        if ($latestmatch -gt $lastcheck) {
            # send your e-mail here
        }
        $lastcheck = $startcheck                    # remember when check was STARTED, not when it finished!!!
                                                    # who knows how long it takes to look for the errors???
        Start-Sleep -Seconds ($cycletime * 60)
    }
}

Answer 3

@Rich Matheisen thank you for your code. I tested it and it doesn't seem to catch the errors in the log that I have or to say it better, it doesn't send me an Email. I have tested our exchage and it is working fine. The error that I am trying to catch is the one that I have previously attached and added in your regex.
Any ideas why?

Thank you in advance for your time and effort!

Answer 4

Does running this return "True"?

$x = "<ERROR><13-Ju1-2020 05:23.21.270> Exception processing TriggerConfigFiles folder :org.xm1.sax.SAXSParseException: Content is not allowed in prolog."
$regex = "\<Error\>\<(.+?\s.+?)\>\s.+Content is not allowed in prolog\.$"
$x -match $regex

The only data I have is what you show me, and a picture of your data doesn't tell me if there are (say) trailing spaces, tabs, etc.

You can try changing the regex to take into account any trailing whitespace:

$regex = "\<Error\>\<(.+?\s.+?)\>\s.+Content is not allowed in prolog\.\s*$"

If it still doesn't work, upload a small TXT file that has some of the errors in it, along with some of the surrounding lines.

Answer 5

@Rich Matheisen thank you very much for your time and effort! It is working! We had some issues with our Exchange, and now that they are solved, I made some tests and I got the Email notification!
Thank you again!