Measure Memory Performance Test (Compact 7)
3/12/2014
The Measure Memory Performance test measures many of the latencies, sizes and properties of the various stages of the memory hierarchy such as L1, L2, SDRAM, and TLB. It does this to highlight the key memory component performance characteristics of a system. It also measures these memory characteristics when RAM is accessed in different modes (cached, un-cached, DirectDraw surface). Then it combines these measurements into a final benchmark score which rates a system on how well its memory characteristics affect a key Windows Embedded Compact End-User scenario.
Note that this test provides information that can be used to monitor memory performance, but does not set a specific threshold for performance that must be obtained.
Test Prerequisites
Your device must meet the following requirements before you run this test.
The following table shows the software requirements for the test.
| Requirements | Description |
|---|---|
Tux.exe |
Test harness, required for executing the test |
Kato.dll |
Logging engine, required for logging test data |
ceperf.dll |
Module containing functions that monitor and log performance. |
perfscenario.dll |
Module containing functions that monitor and log performance. |
MemoryPerf.dll |
Library containing the test cases |
There are no hardware prerequisites for running this test; however, for more consistent results, a simple image with a minimal number of components is recommended.
Subtests
The table below lists the subtests included in this test.
| SubTest ID | Description |
|---|---|
1 |
Cached Memory Performance Test |
2 |
UnCached Memory Performance Test |
3 |
DDrawSurf Memory Performance Test |
4 |
Overall Memory Performance Test: Calculates the overall score using the measurements collected in the above three tests. |
Setting Up the Test
This test has no additional requirements, beyond the standard test environment setup.
Running the Test
Optional Command Line Information :
s tux -o -d MemoryPerf.dll -c "[-c][-f][-h][-l ddd][-o][-p][-r][-s ddd]" -f"release\memoryperf.log"
| Command line parameter | Description |
|---|---|
-c |
Outputs details to memoryperf.csv |
-f |
Fills all allocated buffers with random data before the test |
-h |
Warm Up CPU |
-l ddd |
Sets the priority level for the tests to ddd (default 152, consider 0 if power management is disabled) |
-o |
Prints overview latency info to the csv file |
-p |
Prints additional explanation and diagnostic info to the csv file |
-s ddd |
Sets sleep time at priority changes to ddd ms (default is 1, consider 1 if power management is disabled) |
-w |
Toggles warnings which are enabled by default. |
-2 |
Disables testing for L2 (normally enabled). |
Verifying the Test
When the test completes running, verify that "PASSED" appears in the test log for all sub-tests. A "PASS" test result indicates that the test ran successfully, but the test engineer must manually evaluate the results for each device. Investigate any warning messages that are present in the test log.
Overall Memory Score - "XX Perf Report Overall Memory Score"
Near the bottom of the test log a line similar to this one will be displayed. This score is an aggregation of the device memory performance measures. Take note of a test result when the overall score is significantly less than 100. On an optimized device, the overall score should be >= 100.
The test outputs results via PerfScenario.dll to a "MemoryPerf_test.xml" file, which reports detailed information on the memory performance. This file is generated on the device, and copied back to your desktop by the Test Kit. MemoryPerf_test.xml will be located in the Log File(s) folder that is displayed in the Test Case Results History screen.
After the test is run, examine the MemoryPerf_test.xml to determine the following:
1. Size values should exactly match the silicon design spec of the device.
2. Most ChangeAverage Values are latencies expressed in picoseconds, smaller values indicate higher performance.
3. Locate regressions of the xml "ChangeAverage" values from previous runs or specific design specifications.
4. Regression failures should be investigated and understood.
Troubleshooting the Test
* This test fails to generate results if CEPerf.dll or PerfScenario.dll, dependent logging libraries, are missing.
* This test may not generate the exact same results each time, but should be fairly consistent between different runs.
* If the test fails, determine the point of failure and record the exact error message. Check that the setup steps were followed and that all pre-requisites were met. If the source code is available, examine the point of failure in code to see if any additional information can be gathered about the failure domain.
For additional platform specific issues, consult the CTK articles on the TechNet wiki.