2D/3D Graphics Acceleration (GPU)
Introduction
OpenGL for Embedded Systems, often also known as OpenGL ES or GLES, is a subset of the OpenGL API for 2D and 3D rendering of computer graphics. If your project requires a graphical user interface (GUI), you are advised to use OpenGL ES to have GPU graphics acceleration, either directly or abstracted by a high-level framework.
There are usually two paths to have hardware-accelerated graphics in your project:
- Write your own OpenGL ES application: it gives you more control, but it is the hard way and few customers should follow this path. If you want or rather need to follow this path, this page has some code samples.
- Use a graphics library/framework: choose a framework that supports OpenGL ES graphics acceleration, for example Qt or Crank. It abstracts away the OpenGL API and reduces your time-to-market.
Embedded Linux
Often, to use a graphics framework that has graphics acceleration, you need a graphics back-end. Toradex BSPs support the following graphics back-ends:
- X11: Supported on pre-i.MX 8 Series modules on older BSPs including 4.0. Dropped on BSP 5.0 onwards. However, pre-i.MX 8 Series modules may still optionally support it.
- Wayland: Supported on i.MX 8 Series modules as well as BSPs starting from 5.0. You should prefer to use Wayland whenever possible, but if X support is absolutely required, you may evaluate using the XWayland compatibility layer.
Those graphical back-ends, as well as OpenGL itself, are supported on both Embedded Linux offerings by Toradex:
Frameworks
See some GUI frameworks that support OpenGL ES acceleration:
Validation
Toradex validates OpenGL ES using the Multimedia Reference Image from the Toradex BSP Layers and Reference Images for Yocto Project. We run a simple application to validate that the API works and run the glmark or glmark2 benchmark to get a rough estimate of performance. We use a heatsink to run the tests, otherwise, throttling of the GPU clock may interfere with the test results.
Apalis iMX6
- glmark2, OpenGL ES 2, X11 (
glmark2-es2 -s 640x480) - Apalis iMX6Q score: 397
- Apalis iMX6Q IT score: 324
Apalis iMX8
Two benchmarks are run on the Apalis iMX8QM with specific CPU cores: one on the 4x Arm Cortex-A53 and another on the 2x Arm Cortex-A72 cores.
- HDMI screen, 4k + LVDS display, 1280x800
- glmark2, OpenGL ES 2, Wayland (
taskset -c 0-3 glmark2-es2-wayland -s 640x480 && taskset -c 4,5 glmark2-es2-wayland -s 640x480) - Apalis iMX8QM Cortex-A53 score: 1570
- Apalis iMX8QM Cortex-A72 score: 1427
Apalis iMX8X
Apalis iMX8X is phased out, and it is not available for purchase anymore. The latest supported BSP and TorizonCore version is 5.4.0.
Apalis TK1
OpenGL - NVIDIA downstream-based kernel:
- glmark2, OpenGL, X11 (
glmark2 -s 640x480) - Apalis TK1 score: 1156
OpenGL ES - NVIDIA downstream-based kernel:
- glmark2, OpenGL ES 2, X11 (
glmark2-es2 -s 640x480) - Apalis TK1 score: 1448
OpenGL - upstream-based kernel:
- glmark2, OpenGL, X11 (
glmark2 -s 640x480) glmark2currently does not work.
OpenGL ES - upstream-based kernel:
- glmark2, OpenGL ES 2, X11 (
glmark2-es2 -s 640x480) - Apalis TK1 score: 94
Colibri iMX6
- glmark2, OpenGL ES 2, X11 (
glmark2-es2 -s 640x480) - Colibri iMX6DL score: 197
- Colibri iMX6DL IT score: 208
- Colibri iMX6S IT score: 184
Colibri iMX8X
- glmark2, OpenGL ES 2, Wayland (
glmark2-es2-wayland -s 640x480) - Colibri iMX8QXP score: 929
- Colibri iMX8DX score: 614
Verdin iMX8M Mini
- HDMI screen, 1920x1080
- glmark2, OpenGL ES 2, Wayland (
glmark2-es2-wayland -s 640x480) - Verdin iMX8M Mini score: 366
Verdin iMX8M Plus
Test conditions:
- BSP 5.2.0
- Verdin iMX8M Plus Q 4GB WB IT
- Qt demo stopped
- Without heatsink
- HDMI screen, 1920x1080
Test and result:
- glmark2, OpenGL ES 2, Wayland (
glmark2-es2-wayland -s 640x480) - Verdin iMX8M Plus score: 953
Samples
this section has not been updated since 2017.
If you still want to use the OpenGL ES API directly, this section may contain various code samples.
NVIDIA OpenGL ES 2.0 Samples
The sample programs from NVIDIA are available for download from our Samples page. These samples are only meant for the Tegra 2 and Tegra 3 based modules.
OpenEmbedded Build
You can cross-compile and add the samples to a Linux image using OpenEmbedded.
How to build using OpenEmbedded
Read this article for OpenEmbedded setup.
Add the following line to the local.conf file:
IMAGE_INSTALL_append = "nvsamples"
And build the Toradex image:
bitbake angstrom-lxde-image
Cross-compile From Source
It might be useful to build from source when developing an application.
How to cross-compile from source
The following instructions assume you have the OpenEmbedded SDK environment variables exported.
You need an SDK to cross-compile from source. Read this article to build an SDK for your image using OpenEmbedded.
In your home folder, download the samples and patches. Apply the patches:
wget -c https://developer-archives.toradex.com/files/toradex-dev/uploads/media/Colibri/Linux/Samples/nvsamples.tar.bz2
tar xjvf nvsamples.tar.bz2
cd nvsamples
wget -c https://developer-archives.toradex.com/files/toradex-dev/uploads/media/Colibri/Linux/Samples/nvsamples_armv7ahf_patches.tar.bz2
tar xjvf nvsamples_armv7ahf_patches.tar.bz2
patch -p1 < nvsamples-oe.patch
patch -p1 < nvsamples-no-binary-shaders.patch
patch -p1 < nvsamples-hardfp.patch
Make sure that you have run the patch commands inside the nvsamples directory.
Edit the file make/nvdefs.mk to adjust the NV_TOPDIR samples path:
NV_TOPDIR = $(HOME)/nvsamples
Recompile the auxiliary tools:
cd samples/tools/nvtexfont2
make clean
make
cd ../nvgl2demo_common/
make clean
make
cd ../../../
ln -sf ~/nvsamples/samples/tools/nvtexfont2/libnvtexfont2.a lib-target/
ln -sf ~/nvsamples/samples/tools/nvgl2demo_common/libnvgl2demo_common.a lib-target/
Compile the samples:
cd samples/opengles2/
make clean
make
Copy the textures, shaders and binaries to the target. Change the IP address of the target accordingly:
cd ../../
mkdir textures shaders
cp -a samples/opengles2/ctree/textures/* textures/
cp -a samples/opengles2/bubble/textures/* textures/
cp -a samples/opengles2/ctree/*glsl? shaders/
cp -a samples/opengles2/bubble/*glsl? shaders/
cp -a samples/opengles2/gears/*glsl? shaders/
scp -r textures/ root@192.168.0.2:/home/root
scp -r shaders/ root@192.168.0.2:/home/root
scp samples/opengles2/ctree/ctree root@192.168.0.2:/home/root
scp samples/opengles2/bubble/bubble root@192.168.0.2:/home/root
scp samples/opengles2/gears/gears root@192.168.0.2:/home/root
ARM's OpenGL ES 2.0 Emulator and Samples
ARM provides an OpenGL ES emulator that maps OpenGL ES API calls to the OpenGL API, and it can be found here. It comes without samples, but you can also download the ARM Mali SDK for Linux, that is shipped with samples and can be built to run by the emulator but also on top of devices that support OpenGL ES and X11.
Cross-Compile From Source
It might be useful to build from source when developing an application.
How to cross-compile the samples
The following instructions assume you have the OpenEmbedded SDK environment variables exported.
You need an SDK to cross-compile from source. Read this article to build an SDK for your image using OpenEmbedded.
Download and extract the ARM MALI samples, that are shipped in the ARM Mali SDK for Linux:
cd ~
wget -c https://developer.arm.com/-/media/Files/downloads/mali-sdk/v2.4.4/Mali_OpenGL_ES_SDK_v2.4.4.ef7d5a_Linux_x64.tar.gz
tar xzf Mali_OpenGL_ES_SDK_v2.4.4.ef7d5a_Linux_x64.tar.gz
cd Mali_OpenGL_ES_SDK_v2.4.4
You have to modify some configuration files in order to use the OpenEmbedded cross-toolchain.
In the target cmake file "arm-linux.cmake" below, the GCC_COVERAGE_COMPILE_FLAGS was set based on the CC environment variable set by the target SDK. It may need to be adjusted according to your target (for instance, you can set -mfpu=neon for SoCs that have this co-processor integrated).
...
set (GCC_COVERAGE_COMPILE_FLAGS "-mfpu=vfpv3-d16 -march=armv7-a -mthumb -mfpu=vfp -mfloat-abi=hard $ENV{CFLAGS}")
set (GCC_COVERAGE_LINK_FLAGS $ENV{LDFLAGS})
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${GCC_COVERAGE_COMPILE_FLAGS}")
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${GCC_COVERAGE_COMPILE_FLAGS}")
set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${GCC_COVERAGE_LINK_FLAGS}")
set (CMAKE_SYSROOT "$ENV{TOOLCHAIN_ROOT}/sysroots/armv7at2hf-vfp-angstrom-linux-gnueabi/")
set (CMAKE_CXX_COMPILER "${TOOLCHAIN_ROOT}/sysroots/x86_64-angstromsdk-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-g++")
set (CMAKE_C_COMPILER "${TOOLCHAIN_ROOT}/sysroots/x86_64-angstromsdk-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc")
set (CMAKE_FIND_ROOT_PATH "${TOOLCHAIN_ROOT}/sysroots/armv7at2hf-vfp-angstrom-linux-gnueabi/")
...
It is also required to use the X11 libraries, since the Toradex pre-built image comes with LXDE:
...
else(${TARGET} STREQUAL "arm")
# Linux on ARM
set (GLES_LIB "libGLESv2.so")
set (EGL_LIB "libEGL.so")
set(DUMMY_MALI_LIBRARIES GLESv2 EGL)
add_subdirectory(lib/arm)
set (OPENGLES_LIBRARIES ${DUMMY_MALI_LIBRARIES})
include_directories(simple_framework/inc/mali)
set (PLATFORM_LIBRARIES X11 m)
endif()
...
In the samples directory, modify the cmake file samples/CMakeLists.txt if you want to supress the build of OpenGL ES 3.0 and 3.1 samples. This might be required since some embedded GPUs only support up to OpenGL ES 2.0.
add_subdirectory(opengles_20)
#add_subdirectory(opengles_30)
#add_subdirectory(opengles_31)
We need to tell cmake to use DesktopLinuxPlatform.cpp instead of LinuxOnARMPlatform.cpp. This is done by editing the CMakeLists.txt file inside the simple_framework directory:
...
elseif(${TARGET} STREQUAL "arm")
file(GLOB PLATFORM_SOURCES
src/DesktopLinuxPlatform.cpp
)
endif()
...
It is also required to change some compiler directives for the source-code to use X11 instead of the framebuffer. See the diff comparison between original and modified files, and change yours accordingly:
@@ -19,8 +19,6 @@
{
#if defined(_WIN32)
return WindowsPlatform::getInstance();
- #elif defined(__arm__) && defined(__linux__)
- return LinuxOnARMPlatform::getInstance();
#elif defined(__linux__)
return DesktopLinuxPlatform::getInstance();
#endif
@@ -83,14 +83,7 @@
typedef void *EGLNativeWindowType;
typedef void *EGLNativePixmapType;
-#elif defined(__arm__) && defined(__gnu_linux__) /* ARM Linux Mali */
-#include <EGL/fbdev_window.h>
-
-typedef void* EGLNativeDisplayType;
-typedef void* EGLNativePixmapType;
-typedef fbdev_window* EGLNativeWindowType;
-
-#elif defined(__unix__)
+#elif defined(__linux__)
/* X11 (tentative) */
#include <X11/Xlib.h>
@@ -42,8 +42,6 @@
#if defined(_WIN32)
HWND window;
HDC deviceContext;
- #elif defined(__arm__) && defined(__linux__)
- fbdev_window *window;
#elif defined(__linux__)
Window window;
Display* display;
@@ -83,9 +81,6 @@
#if defined(_WIN32)
#include "WindowsPlatform.h"
-#elif defined(__arm__) && defined(__linux__)
-#include "LinuxOnARMPlatform.h"
-
#elif defined(__linux__)
#include "DesktopLinuxPlatform.h"
@@ -219,9 +219,6 @@
#if defined(_WIN32)
/* Win32 */
display = eglGetDisplay(platform->deviceContext);
-#elif defined(__arm__)
- /* Linux on ARM */
- display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
#else
/* Desktop Linux */
platform->display = XOpenDisplay(NULL);
@@ -278,14 +275,9 @@
* On ARM devices perform a strict match to ensure we get the best performance.
* On desktop devices perform a loose match to ensure greatest compatability.
*/
-#if defined(__arm__)
- config = findConfig(true);
-#else
+#if defined(__linux__)
config = findConfig(false);
-#endif
-
-#if defined(__linux__) && !defined(__arm__)
- ((DesktopLinuxPlatform*)(platform))->createX11Window();
+ ((DesktopLinuxPlatform*)(platform))->createX11Window();
#endif
/* Create a surface. */
Create a directory for the build and run cmake inside it:
mkdir build_arm
cd build_arm
cmake -G "Unix Makefiles" -DTARGET=arm ..
make install
Copy the application to the target. Change the IP address of the target accordingly:
scp -r ../bin_arm root@192.168.0.2:/home/root
You must run each application inside its own directory. Otherwise it will not find the shaders, etc...
NXP (Formerly Freescale) i.MX 6 Graphics SDK
NXP (formerly Freescale) provides GPU SDKs shipped with OpenGL ES and OpenVG samples. You can build the samples from it for use on top of X11.
There are two versions of the GPU SDK, and both of them are available here, being:
- The old SDK under the "Snippets, Boot Code, Headers, Monitors, etc." section and;
- The new SDK under the "Software Development Kits" section once clicking "More".
At the time of writing, The old SDK revision is 1.0.0 (dated 01/25/2013) and the new SDK revision is 2.3 (dated 02/23/2016).
Cross-Compile Using the old SDK
The old SDK has OpenGL ES 1.1 and 2.0, and OpenVG samples. It is a Makefile based project with X11 and framebuffer support. Below are provided instructions on how to build the samples for X11.
How to cross-compile using the old SDK
The following instructions assume you have the OpenEmbedded SDK environment variables exported.
You need an SDK to cross-compile from source. Read this article to build an SDK for your image using OpenEmbedded.
Download the gpu_sdk_v1.00.tar.gz tarball from NXP/Freescale's website, under the Snippets, Boot Code, Headers, Monitors, etc. section: Software & Tools.
Create a directory and extract the tarball.
cd
mkdir gpu_sdk_v1.00
cd gpu_sdk_v1.00
tar xf ../gpu_sdk_v1.00.tar.gz
We're going to build the samples contained in the subdirectory 'Samples'.
The makefiles create executables for the softfp float ABI which is incompatible with our images. Thus change the CFLAGS in all makefiles to allow setting the ABI from the outside and also pass this information in the link step. Also, instead of unconditionally setting the CC and AR variables, allow the use of environment exported variables, for us to use the environment variables exported by the SDK.
cd; cd gpu_sdk_v1.00/Samples
find . -name "Makefile*" -exec sed -i 's/softfp/\$\(FLOATABI\)/' {} \;
find . -name "Makefile*" -exec sed -i 's/\-lstdc/\$\(CFLAGS_TUT\) -lstdc/' {} \;
find . -name "Makefile*" -exec sed -i 's/CC\ \=/CC\ ?\=/' {} \;
find . -name "Makefile*" -exec sed -i 's/AR\ \=/AR\ ?\=/' {} \;
Use the toplevel makefiles to set the float ABI. Add the following line to the top of gpu_sdk_v1.00/Samples/GLES1.1/Makefile.x11 and gpu_sdk_v1.00/Samples/GLES2.0/Makefile.x11:
export FLOATABI = hard
OpenGL ES 1.1
The top-level makefile references two unavailable subprojects 18_VertexBufferObjects and 19_Beizer. Remove them from the SUB_PROJECTS variable.
Also, add the following line to the includes of main.h, for the 06_Projection, 11_LightingFog and 15_ParticleAccelerator samples. Otherwise, "‘GLshort’ does not name a type" error will be raised during compilation time:
#include <GLES/glext.h>
Make sure you have the SDK environment variables CC and AR set. Build the examples as follows:
cd; cd gpu_sdk_v1.00/Samples/GLES1.1
make -f Makefile.x11
make -f Makefile.x11 install
The executables can be found in gpu_sdk_v1.00/Samples/GLES1.1/bin/GLES11_x11/ ready for copying to your module.
OpenGL ES 2.0
The sample 05_PrecompiledShader does not link because the linker can not directly link the function glProgramBinaryOES. Adding the following to Samples/GLES2.0/05_PrecompiledShader/PrecompiledShader.c fixes this:
@@ -99,3 +99,5 @@ static GLuint LoadProgram(const char *programName)
long flength;M
+ // get the glProgramBinaryOES function
+ PFNGLPROGRAMBINARYOESPROC glProgramBinaryOES = (PFNGLPROGRAMBINARYOESPROC)eglGetProcAddress("glProgramBinaryOES");
// create the program objectM
Make sure you have the SDK environment variables CC and AR set. Build the examples as follows:
cd; cd gpu_sdk_v1.00/Samples/GLES2.0
make -f Makefile.x11
make -f Makefile.x11 install
The executables can be found in gpu_sdk_v1.00/Samples/GLES2.0/bin/GLES20_x11/ ready for copying to your module.
OpenVG
The X11 makefiles for the OpenVG samples are missing. Add them with the following patch:
cd; cd gpu_sdk_v1.00/Samples
wget https://docs.toradex.com/102840-openvg_x11_patch.bz2
tar xf 102840-openvg_x11_patch.bz2
patch -p 1 < openvg_x11.patch
Make sure you have the SDK environment variables CC and AR set. Build the examples as follows:
cd; cd gpu_sdk_v1.00/Samples/OpenVG
make -f Makefile.x11
make -f Makefile.x11 install
The executables can be found in gpu_sdk_v1.00/Samples/OpenVG/bin/OpenVG_x11/ ready for copying to your module.
Cross-compile using the new SDK
The new SDK has OpenGL ES 2.0 and 3.0, and OpenVG samples. It has its Makefiles generated by Python scripts and supports Android SDK+NDK, Ubuntu, Windows and Yocto builds, being the latter possible for X11, framebuffer and Wayland. Below are provided instructions on how to build the samples for X11, using OpenEmbedded/Yocto.
How to cross-compile using the new SDK
You need an SDK to cross-compile from source. Read this article to build an image and SDK using OpenEmbedded. You will need to add additional packages to the standard recipe provided by Toradex at later steps.
Download the fsl-gpu-sdk-2.3.zip zip file from NXP/Freescale's website, under the Software Development Kits section once clicking More: Software & Tools.
Create a directory and extract the tarball following the instructions provided by the Readme.txt file, accepting the Freescale/NXP license:
unzip fsl-gpu-sdk-2.3.zip
chmod a+x fsl-gpu-sdk-2.3.bin
./fsl-gpu-sdk-2.3.bin
rm Readme.txt fsl-gpu-sdk-2.3.bin fsl-gpu-sdk-2.3.zip
Add dependencies to the OpenEmbedded build
Enter the SDK directory. Extract the OpenEmbedded layer meta-gtec and add it to your OpenEmbedded build. The instructions below are based in the documentation provided with the SDK (Doc/Setup_guide_yocto.txt).
cd; cd fsl-gpu-sdk-2.3
tar xzf meta-gtec.tar.gz
mv meta-gtec <your-openembedded-layers-directory>
Add the layer to bblayers.conf and the packages assimp and devil to local.conf:
...
BBLAYERS ?= " \
${TOPDIR}/../layers/meta-gtec \
...
...
IMAGE_INSTALL_append = "assimp devil"
Build the angstrom-lxde-image and SDK. Flash the image to the iMX6 computer on module and install the SDK.
OpenGL ES 2, OpenGL ES 3 and OpenVG samples
Export the SDK and the ROOTFS and FSL_PLATFORM_NAME environment variables as described below, and source the prepare.sh script:
. <path-to-the-sdk-folder>/environment-setup-armv7at2hf-neon-angstrom-linux-gnueabi
export ROOTFS=<path-to-the-sdk-folder>/sysroots/armv7at2hf-neon-angstrom-linux-gnueabi
export FSL_PLATFORM_NAME=Yocto
. prepare.sh
There are some files that are lacking the cmath include, please add the following line to the files:
#include <cmath>
Files:
gedit ./DemoFramework/FslDemoHost/source/FslDemoHost/DemoAppProfilerOverlay.cpp ./DemoFramework/FslSimpleUI/source/FslSimpleUI/Control/CheckBox.cpp ./DemoFramework/FslGraphics3D/source/FslGraphics3D/Camera/ArcballCamera.cpp ./DemoFramework/FslGraphics_Draft/source/FslGraphics_Draft/Procedural/TorusGenerator.cpp ./DemoApps/Shared/FractalShaderShared/source/FractalShaderShared/ShaderUtil.cpp ./DemoApps/Shared/FractalShaderShared/source/FractalShaderShared/JuliaHelper.cpp ./DemoApps/Shared/EnvironmentMappingShared/source/EnvironmentMappingShared/SphereMeshCreator.cpp ./DemoApps/Shared/DFNativeBatch2DShared/source/DFNativeBatch2DShared/TextureAtlasScene.cpp ./DemoApps/Shared/DFGraphicsBasic2DShared/source/DFGraphicsBasic2DShared/Shared.cpp ./DemoApps/OpenVG/BitmapFont/source/BitmapFont.cpp ./DemoApps/GLES3/T3DStressTest/source/T3DStressTest.cpp ./DemoApps/GLES3/T3DStressTest/source/FurTexture.cpp ./DemoApps/GLES2/T3DStressTest/source/FurTexture.cpp ./DemoApps/GLES3/FurShellRendering/source/FurTexture.cpp ./DemoApps/GLES3/FurShellRendering/source/FurShellRendering.cpp ./DemoApps/GLES3/FractalShader/source/FractalShaderMandelbrot.cpp ./DemoApps/GLES3/E3_0_InstancingSimple/source/E3_0_InstancingSimple.cpp ./DemoApps/GLES3/T3DStressTest/source/T3DStressTest.cpp ./DemoApps/GLES2/T3DStressTest/source/T3DStressTest.cpp ./DemoApps/GLES2/FractalShader/source/FractalShaderMandelbrot.cpp ./DemoApps/GLES2/Blur/source/GausianHelper.cpp
To compile all the samples:
./build.sh -f GNUmakefile_Yocto -j 16 EGLBackend=X11
The OpenGL ES 2, OpenGL ES 3 and OpenVG executables are inside the DemoApps directory.
Windows CE
One possible framework to abstract the direct use of OpenGL ES API is Qt: Setting up Qt framework with VS2008 for WinCE with Toradex modules.
Samples
Simple OpenGL ES samples:
The Nvidia Khronos SDK contains samples for OpenGL ES1, OpenGL ES2, OpenMax and OpenVg.