Edit

Share via

Standard Test Metadata

  • Article

The following Test 'mark-up' metadata is standard metadata that can be applied to TAEF tests.

Implicit Metadata

Certain pieces of metadata are automatically inferred from the markup of the tests:

  • "Name" - the fully qualified name of the test.
  • "Architecture" - the processor architecture of the DLL. This value will be one of 'x86', 'x64' or 'arm'.
  • "TestFile" - The DLL file that the test was described in.

Selection Metadata

The selection metadata are simply 'preferred' pieces of metadata to allow teams to have a standard to allow them to better consume one another's tests. There is no required metadata - mandating metadata increases the cost of adding automation, and all metadata should be optional, or should enable 'opt-in' behavior.

There are cases when multiple values can be specified for a metadata value, in which case you should use a semicolon-separated list, and use a 'contains' style selection query, to test for it. For example, if the "Owner" metadata needs two values, then it should be set to "Someone;SomeoneElse". The query to select tests that are owned only by Someone would be:

te Wex.Common.Tests.dll /select:@Owner='Someone'

Whereas, the following query would select tests that are owned or co-owned by Someone:

te Wex.Common.Tests.dll /select:@Owner='*Someone*'

You can define your own metadata to use within your own company. The following suggestions are recommendations. .

"You should..." Metadata

These metadata properties are recommendations and have clear meanings. Use these metadata properties as you need them:

"ActivationContext"
Specifies a particular version of binary from various side-by-side assemblies in the system. See Activation Context for details.

"BinaryUnderTest"
The binary that a given test is [unit] testing. This allows developers to quickly run all unit tests that verify a given DLL.

"DefaultTestResult"
Overrides the default test result of "Passed" for the given test. If the test passes, the logged result will be the default test result. Possible values are "Passed", "Failed", "NotRun", "Blocked" and "Skipped".

"DeploymentItem"
Identifies files and folders as test dependencies.

"Description"
A short description of what the test does.

"DpiAware"
When set to "true," TAEF will run your tests in a process marked as DPI-aware, see High DPI.

"ExecutionGroup"
A set of consecutive tests within a class that need to be run in order and are blocked if a previous test in the execution group is not run or fails. See Execution Groups for details.

"Ignore"
Test classes or test methods with "Ignore" metadata set to "true" are skipped during execution or listing by TAEF. To override this behavior and run or list all tests including the ones with "Ignore" metadata, specify /runIgnoredTests as a command-line argument.

"IsolationLevel"
Specifies the minimum level of isolation to be used when executing TAEF tests. See Test Isolation for more details.

"Parallel"
Executes tests in parallel across multiple processors. For more details, see Parallel.

"Priority"
The priority of the test as an integer, smaller is high-priority.

"RebootPossible"
When set to true, enables the use of the Reboot APIs to request TAEF perform a computer restart or inform TAEF of an impending test-initiated restart.

"RunAs"
Specifies the context in which the tests in concern should be run. See RunAs Execution for details.

"RunFixtureAs"
Specifies the context in which the test fixtures in concern should be run. See RunFixtureAs for details.

"TestClassification:Scope"
The Test Classification "Scope" identifies the test collateral used to validate "engineering process events" that occur in Windows.

"TestClassification:Type"
Test Classification "Type" identifies the types of tests that need to be distinguished.

"TestClassification"
Use property value "Unit:WUTG" to indicate a unit test that conforms to Windows Unit Testing Guidelines (WUTG). Use property value "Unit:WUTG:ChexGate" to indicate a unit test that conforms to Windows Unit Testing Guidelines (WUTG) and should run during gated phase of Chex scenario (failure blocking submit).

"TestTimeout"
Specifies the maximum amount of time a given test or setup/cleanup method can take. See Timeouts for details.

"ThreadingModel"
The pre-configured COM threading model used by the test. See Configuring Threading Models for details.

Data-driven testing related:

"DataSource"
Specifies the main source for data for data-driven tests.

"TableId"
Specifies the name or Id of the Table separate from the "DataSource" in case of Table-Based Data-driven tests.

"Pict:Timeout" (and deprecated "PictTimeout")
Overrides the default time-out of 5 minutes allowed for PICT.exe to process the user specified model file in case of PICT based data-driven tests.

"Pict:SeedingFile" (and deprecated "Seed")
Specified the relative location to the seed file, separate from the "DataSource" in case of PICT based data-driven tests.

"Pict:Order"
Specifies the value of the /o parameter for PICT.exe when it is called in PICT based data-driven tests.

"Pict:ValueSeparator"
Specifies the value of the /d parameter for PICT.exe when it is called in PICT based data-driven tests.

"Pict:AliasSeparator"
Specifies the value of the /a parameter for PICT.exe when it is called in PICT based data-driven tests.

"Pict:NegativeValuePrefix"
Specifies the value of the /n parameter for PICT.exe when it is called in PICT based data-driven tests.

"Pict:Random"
Specifies whether randomness should be used when calling PICT.exe for PICT based data-driven tests. When this is true, the random seed that was used is logged by TAEF.

"Pict:RandomSeed"
Specifies the value of the /r parameter for PICT.exe when it is called in PICT based data-driven tests. Setting this changes the default for "Pict:Random" from false to true.

"Pict:CaseSensitive"
Specifies whether the /c parameter should be used for PICT.exe when it is called in PICT based data-driven tests.

Support for Device related:

"TestResourceDependent"
Specifies that the tests in current scope are dependent on the TestResource and function on the resources collected by BuildResourceList(...). See Support for Devices for details.

"ResourceSelection"
Specifies the query to match TestResources collected by BuildResourceList(...) which are relevant for the tests in question. See Support for Devices for details.

"You can..." Metadata

These metadata properties can be used, but their interpretation is not guaranteed; teams can use them if they want to.

"Owner"
The alias of the owner of the test.

"ProcessUnderTest"
Useful for runtime analysis. For example, if a test is testing "Explorer.exe", then run Radar (a runtime analysis tool) against the process.

"Feature"
An identifier that categorizes the test to a specific feature or technology. This should be treated as a 'cookie' identifier who's interpretation is down to the team that defines it.

'Reserved' Metadata

The following metadata may be used in the future - please don't use it.

  • User
  • IntegrityLevel
  • Timeout
  • HostType