Skip to main content
Version: Torizon OS 7.x.y

How to use GStreamer on Torizon OS

Introduction​

Video processing has become a major feature in the embedded world and has many facets. From decoding a camera's video stream to pre-process frames for an AI application, the use of GStreamer is a must. Even more so if we can take advantage of Hardware Acceleration provided by the System on Module's Video Processing Unit (VPU). This article describes how to use GStreamer on Torizon OS to create simple pipelines. For iMX8 based SoMs, it also shows how to leverage hardware acceleration provided by the VPU in GStreamer pipelines.

For a comprehensive overview of GStreamer, see Video Processing - GStreamer.

The article Video Encoding and Playback (Linux) is not specific for Torizon, but some instructions may be applicable. For example, generic information about Wayland/Weston is provided.

The article How to use Cameras on Torizon aims to help you with instructions about the configuration and usage of cameras in the context of containers, using Gstreamer pipelines and Video4Linux2.

NXP provides GStreamer plugins to access the multimedia libraries using the i.MX SoC's hardware acceleration units. Torizon OS helps to reduce the complexity of the setup with Debian packages providing these HW optimized GStreamer plugins.

This article complies with the Typographic Conventions for Torizon Documentation.

Prerequisites​

For iMX8 based System on Module (SoM), it's recommended to use the Torizon IDE Extension.

Wayland + Weston​

This example will run a pipeline that uses waylandsink GStreamer's plugin. This plugin runs on top of Wayland and Weston. It will be necessary to start 2 containers: One with the Weston image, and one with the application image, with Wayland support. Both will communicate through shared folders by bind mounting.

Dockerfile instructions​

The implementation details will be explained in this session. See the Quickstart Guide with the instructions about how to compile the image on a host pc and pull the image onto the board. You can also scp this file to the board and build it locally.

Building the Container Image​

caution

Make sure you have configured your Build Environment for Torizon Containers.

Now it's a good time to use torizon-samples repository or download the sample Dockerfile:

$ cd ~
$ git clone --branch bookworm https://github.com/toradex/torizon-samples.git
$ cd ~/torizon-samples/gstreamer/bash/simple-pipeline

And build the container image:

$ docker build --build-arg BASE_NAME=wayland-base-vivante --build-arg IMAGE_ARCH=linux/arm64/v8 -t <your-dockerhub-username>/gst_example .

After the build, push the image to your Dockerhub account:

$ docker push <your-dockerhub-username>/gst_example

Details about the Dockerfile​

In this section, you will go through some important snippets containing information about the Dockerfile.

Toradex Debian image - Wayland​

Toradex provides a basic image with Wayland libraries on its Dockerhub page.

Ensure you use a the container image and tag compatible with your Torizon OS version.

Select the correct image for your module from the tabs below:

Dockerfile
FROM --platform=linux/arm64/v8 torizon/wayland-base-vivante:3

Install GStreamer application and plugins​

Install the required packages on the image with the following commands:

caution
VPU is not supported with iMX8 Modules using the upstream container.
Dockerfile
RUN apt update
RUN apt install -y libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio

Test GStreamer​

We will set gst-launch-1.0 as entrypoint for our Dockerfile with a default pipeline. A test source pattern will be output to a Wayland window.

Dockerfile
ENTRYPOINT ["gst-launch-1.0"]

CMD ["videotestsrc", "!", "videoconvert", "!", "videoscale", "!", "waylandsink", "sync=false"]

You can modify the pipeline for your needs by adding it after the container name in the docker run command. For example:

docker run -it --rm -v /tmp:/tmp -v /dev:/dev <your-username>/gst_example videotestsrc ! videoconvert ! videoscale ! waylandsink sync=false

Docker-compose​

After building the Dockerfile image above and pushing it to your Dockerhub, you can launch Weston and Wayland with docker-compose.

Select your architecture from the tabs:

The docker-compose file for arm64v8 is available on GitHub.

Still on the torizon-samples/gstreamer/bash/simple-pipeline directory, edit the docker-compose.arm64.yaml file by filling the wayland's service image field with your dockerhub username:

wayland:
  depends_on:
    weston:
      condition: service_healthy
  image: <your-username>/gst_example
  volumes:
      - /tmp:/tmp
      - /sys:/sys
      - /dev:/dev
  restart: on-failure

After filling it, send it to your module using scp:

$ scp docker-compose.arm64.yaml torizon@<SoM-ip>:/home/torizon

Now enter your module using SSH:

$ ssh torizon@<SoM-ip>
info
For more information about SSH, please refer to SSH on Linux.

Now you can launch by using the command:

# docker-compose -f docker-compose.arm64.yaml up


Send Feedback!