Share via

System.ApplicationLog.GenericCSVLog.FilteredEventProvider

  • Article

Applies To: System Center 2012 - Operations Manager, System Center 2012 R2 Operations Manager, System Center 2012 SP1 - Operations Manager

The System.ApplicationLog.GenericCSVLog.FilteredEventProvider data source module type is used to provide filtered events from a non-delimited, text-based, log file and output the events as System.Event.Data.

Usage

Use the module to provide generic log entry data within event data from within a workflow. A CSV generic log file is a file that can contain up to 30 delimited strings per log entry. If your log file includes only non-delimited text within each entry, use the System.ApplicationLog.GenericLog.FilteredEventProvider module type.

Type Definition

<DataSourceModuleType ID="System.ApplicationLog.GenericCSVLog.FilteredEventProvider" Accessibility="Public">
  <Configuration>
    <IncludeSchemaTypes>
      <SchemaType>System!System.ExpressionEvaluatorSchema</SchemaType>
    </IncludeSchemaTypes>
    <xsd:element name="LogFileDirectory" type="xsd:string" />
    <xsd:element name="LogFilePattern" type="xsd:string" />
    <xsd:element name="LogIsUTF8" type="xsd:string" />
    <xsd:element name="Separator" type="xsd:string" />
    <xsd:element name="Expression" type="ExpressionType" />
  </Configuration>
  <ModuleImplementation>
    <Composite>
      <MemberModules>
        <DataSource TypeID="System.ApplicationLog.GenericLogReader" ID="DataSource">
          <LogFileDirectory>$Config/LogFileDirectory$</LogFileDirectory>
          <LogFilePattern>$Config/LogFilePattern$</LogFilePattern>
          <LogIsUTF8>$Config/LogIsUTF8$</LogIsUTF8>
        </DataSource>
        <ConditionDetection TypeID="System!System.ExpressionFilter" ID="Filter">
          <Expression>$Config/Expression$</Expression>
        </ConditionDetection>
        <ConditionDetection TypeID="System!System.Event.GenericDataMapper" ID="Mapper">
          <EventOriginId>$Target/Id$</EventOriginId>
          <PublisherId>$MPElement$</PublisherId>
          <PublisherName>GenericLog</PublisherName>
          <Channel>GenericLog</Channel>
          <LoggingComputer />
          <EventNumber>0</EventNumber>
          <EventCategory>3</EventCategory>
          <EventLevel>0</EventLevel>
          <UserName />
          <Params>
            <Param>$Data/Params/Param[1]$</Param>
          </Params>
        </ConditionDetection>
      </MemberModules>
      <Composition>
        <Node ID="Mapper">
          <Node ID="Filter">
            <Node ID="DataSource" />
          </Node>
        </Node>
      </Composition>
    </Composite>
  </ModuleImplementation>
  <OutputType>System!System.Event.Data</OutputType>
</DataSourceModuleType>

Parameters

The System.ApplicationLog.GenericCSVLog.FilteredEventProvider module supports the configuration parameters described in the following table.

Parameter Type Description

LogFileDirectory

String

Required parameter. Contains the full path to the local directory from which to read the log file or files.

LogFilePattern

String

Required parameter. Contains the file name or wildcard pattern to match for the log files.

LogFileDirectory

BooleanBoolean

Required parameter. Indicates whether the log file is UTF8-encoded. The value should be either true or false.

Expression

ExpressionType

Required parameter. Contains the filter expression to apply to the log file entries.

Expression

The expression that is specified in this parameter is used to filter the log file data that is output from the log reader module. The data type that is passed to the expression module built into the System.ApplicationLog.GenericCSVLog.FilteredEventProvider module type is of type System.ApplicationLog.GenericLogEntryData.

The following code shows an example expression that looks for a substring in the contents of the log file entry:

<Expression>
  <RegExExpression>
    <ValueExpression>
      <XPathQuery Type="String">Params/Param[1]</XPathQuery>
    </ValueExpression>
    <Operator>ContainsSubstring</Operator>
    <Pattern>Warning:</Pattern>
  </RegExExpression>
</Expression>

In the preceding example, only entries that contain “Warning:” in their entry are output by the module.

For more information about the Expression parameter, see ExpressionType.

Composition

The System.ApplicationLog.GenericCSVLog.FilteredEventProvider module is a composite module that contains the member modules described in the following table.

Workflow Run Order Module Type Usage

1

System.ApplicationLog.GenericCSVLogReader

Reads the log file and outputs System.ApplicationLog.GenericLogEntryData data.

2

System.ExpressionFilter

Evaluates whether the specified expression on the incoming data evaluates to true and, if so, passes it on to the consequent mapping module.

3

System.Event.GenericDataMapper

Maps the incoming System.ApplicationLog.GenericLogEntryData data to System.Event.Data data.
Module Type Usage

System.ApplicationLog.GenericCSVLog.EventProvider

Maps delimited, text-based generic log entry data to event data. Returns System.Event.Data data.

System.ApplicationLog.GenericLog.EventProvider

Maps non-delimited, text-based generic log entry data to event data. Returns System.Event.Data data.

System.ApplicationLog.GenericLog.FilteredEventProvider

Maps filtered non-delimited, text-based generic log entry data to event data. Returns System.Event.Data data.

External Module References

None.

Sample

The following sample illustrates how a rule can be implemented to retrieve generic log entry data from any number of log files in a directory. After the module maps the log entry data to event data, the module uses the expression to filter out data that does not contain “WARNING:” in the first column and does not contain “ERROR:” in the second column. After filtering, the data is returned and written to the Operations Manager database and data warehouse.

<Rule ID="Microsoft.Samples.LogEntryToEvent" Target="Microsoft.Samples.TheApplication" Enabled="true">
 <DataSources>
    <DataSource ID="LogDS" TypeID="AppLog!System.ApplicationLog.GenericCSVLog.FilteredEventProvider ">
      <LogFileDirectory>C:\Logs</LogFileDirectory>
      <LogFilePattern>*.log</LogFilePattern>
      <LogIsUTF8>false</LogIsUTF8>
      <Separator>,</Separator>
      <Expression>
        <And>
          <Expression>
            <RegExExpression>
              <ValueExpression>
                <XPathQuery Type="String">Params/Param[1]</XPathQuery>
              </ValueExpression>
              <Operator>ContainsSubstring</Operator>
              <Pattern>Warning:</Pattern>
            </RegExExpression>
          </Expression>
          <Expression>
            <RegExExpression>
              <ValueExpression>
                <XPathQuery Type="String">Params/Param[2]</XPathQuery>
              </ValueExpression>
              <Operator>ContainsSubstring</Operator>
              <Pattern>Error:</Pattern>
            </RegExExpression>
          </Expression>
        </And>
      </Expression>
    </DataSource>

  </DataSources>
  <WriteActions>
    <WriteAction ID="WriteToDB" TypeID="SCLibrary!Microsoft.SystemCenter.CollectEvent"/>
    <WriteAction ID="WriteToDW" TypeID="SCDW!Microsoft.SystemCenter.DataWarehouse.PublishEventData"/>
  </WriteActions>

</Rule>

If the log entry line is WARNING: This is a single log entry., ERROR: This is a single log entry., the output of the System.ApplicationLog.GenericCSVLog.FilteredEventProvider module would be something like the following:

<DataItem type="System.Event.Data" time="2008-03-11T03:08:55.0343534-08:00" sourceHealthServiceId="0A0800A0-A802-E90B-6045-D961D516CA78">
  <EventOriginId>513C3E6D-374A-47d1-94B6-920EB54F9A27</EventOriginId>
  <PublisherId>513C3E6D-374A-47d1-94B6-920EB54F9A27</PublisherId>
  <PublisherName>GenericLog</PublisherName>
  <Channel>GenericLog</Channel>
  <LoggingComputer />
  <EventNumber>0</EventNumber>
  <EventCategory>3</EventCategory>
  <EventLevel>0</EventLevel>
  <UserName />
  <Params>
    <Param>$Data/Params/Param[1]$</Param>
  </Params>
  <EventData>
    <DataItem type="System.ApplicationLog.GenericLogEntryData" time="2007-05-22T10:07:41.6416394-07:00" sourceHealthServiceId="A9BB62D3-BEF2-5208-E680-1EE489235408">
      <LogFileDirectory>c:\Logs</LogFileDirectory>
      <LogFileType>Generic CSV Log File Format</LogFileType>
      <LogFileName>c:\Logs\some.log</LogFileName>
      <Params>
        <Param>WARNING:This is a single log entry.</Param>
        <Param>ERROR:This is a single log entry.</Param>
        </Params>
      <DataItem>
  </EventData>
</DataItem>

Information

   

Module Type

DataSourceModuleType

Input Type

None

Output Type

System.Event.Data

Implementation

Composite

Library

System.ApplicationLog.Library