The benchmark tests have the purpose to provide an estimated comparision of the performance across the modules and BSPs.
Phoronix Test Suite is a framework that integrates benchmarks, executing disk, RAM, system, graphics, network and processor tests. The software is designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner.
The following benchmark tests available on Phoronix Test Suite were selected to measure performance on specific day-to-day operations.
A FLAC is an audio coding format for lossless compression of digital audio. Digital audio compressed by FLAC's algorithm can typically be reduced to between 50 and 70 percent of its original size. The Encode Flac benchmark measures how long it takes to encode a sample WAV file to FLAC format.
This test times how long it takes to encode this WAV file, of 82MB, to FLAC format five times and take the average time.
GtkPerf is an application designed to test GTK+ performance. The application creates common testing platform to run GTK+ widgets, opening comboboxes, toggling buttons, scrolling text, etc. More information here.
This benchmark calculates the average frame-rate from the GtkCmboBox and GtkComboBoxEntry operations, although more tests are available, both operations mentioned before contains 10 entrys "Selection 1"..."Selection 10", the test opens and closes the respectives GtkComboBoxes a certain number of times while selecting next entry.
The STREAM benchmark is a simple, synthetic benchmark designed to measure sustainable memory bandwidth (in MB/s) and a corresponding computation rate for four simple vector kernels (Copy, Scale, Add and Triad), more information here.
This test runs on each core of a mode using the OpenMP programming paradigm. All the four synthetic computational kernels are run, and the best rates observed are reported for each choice of thread count.
SmallPT is a small Monte Carlo path tracer, and generates a single image of a modified Cornell box rendered with full global illumination (GI). It does this by solving the rendering equation, including ideal Lambertian, specular, and dielectric reflection, area lights, anti-aliasing, importance sampling, and multi-threading. It is only 99 lines of C++, but illustrates photo realistic rendering in a correct, and concise manner, more information here.
This test is a global illumination renderer, it renders this scene using unbiased Monte Carlo path tracing using 5000 samples per pixels as argument.
Simple raytracer designed to test the floating-point CPU performance. The default data set is very small, so C-ray really only tests the pure floating-point speed of a CPU core (or multiple CPUs/cores using the threaded version), ie. RAM speed is not a significant factor.
This test is multi-threaded (16 threads per core), will shoot 8 rays per pixel for anti-aliasing, and will generate a 1600 x 1200 image.
In this section the method for reproducing the benchmarks is presented.
Warning: the scripts provided for running the benchmarks are provided as-is and are not maintained.
There is a set of shell scripts that were created to install all dependencies and run the tests. These scripts can be downloaded here as a zip file.
After downloading the zip file into the module, run the "install_dependencies.sh" file. This will install the benchmarks dependencies and the Phoronix Test Suite framework.
Note: Since Phoronix Test Suite uses a considerable amount of flash memory for an embedded system, on some modules it may be necessary to boot from an external flash/storage device. This was the case for the Colibri VF50/VF61 and Colibri iMX7S. For information about different boot scenarious refer to this article. For such cases, it is necessary to make sure that flash/disk benchmarks are run targeting the internal SoC flash storage device.
Next, install all the benchmark tests that were mentioned above. To do this run the "install_tests.sh". If you get any error while installing, run the "install_gtk_encode.sh" file.
All benchmark runs were done with DVFS disabled and the processor clock speed set to maximum. To reproduce this, run the "dvfs_setup.sh" file and choose the module you are using.
Finally, run the "run_tests.sh" script to start the benchmarks. After finishing, all the files will be deployed in a folder called "results" in the home directory.
The Benchmark results of all runs on our modules are available here as a xls file.
The Colibri iMX6S 256MB V1.0A was chosen as the standard SoM for comparison, therefore its result indexes are all "1.0", and the other Computer on Modules results are relative to the Colibri iMX6S 256MB V1.0A.
The below graph presents a summary of the benchmark results: