Skip to main content
Version: 5.0

How to use Cameras on Torizon

Introduction

This article aims to provide helpful information for the first steps with Cameras on TorizonCore. So, it means that this article is going to guide you with the specific details and considerations that surround the usage of a video capture device using containers.

The main goal of using Torizon and containers is to simplify the developer's life and keep the development model close to one of desktop or server applications. However, some requirements inherent to embedded systems development may be considered in the process. Therefore, running a video capture application inside a container requires some aspects like hardware access, running multiple containers, and others. You can read more about these requirements in the section Container Requirements

If you want to learn more about the Torizon best practices and container development workflow for embedded systems applications, you can refer to Torizon Best Practices Guide. Developing applications for Torizon can be done using command-line tools or, as we recommend, the Visual Studio Code Extension for Torizon.

This article workflow is valid for USB Cameras with USB Video Class (UVC) standard driver and some CSI cameras. To find more about USB cameras you can refer to Webcam (Linux). We recommend using one of our supported CSI cameras which you can find at Camera on Toradex Computer on Modules. The process described here regarding the use of a MIPI-CSI camera was done with the CSI Camera Set 5MP AR0521 Color. To use the AR0521 camera you have to enable a Device-Tree Overlay, see more at First Steps with CSI Camera Set 5MP AR0521 Color (Torizon).

info

This article does not cover the usage of IP cameras, you can find information about this topic at Audio/Video over RTP With GStreamer (Linux).

This article complies with the Typographic Conventions for the Toradex Documentation.

Prerequisites

Camera usage with containers

Running your application in a container means that it will run in a "sandboxed" environment. This fact may limit its interaction with other components in the system. Hardware access and the use of multiple containers, mainly focused on the usage of a Graphical User Interface (GUI), are the most important topics that you must know to have a good understanding of how to proceed with camera usage. You can read more about multiple containers at Using Multiple Containers with TorizonCore.

Hardware Access and Shared Resources

Docker provides ways to access specific hardware devices from inside a container, and often one needs to grant access from both a permission and a namespace perspective.

  • Permissions: To grant GPU access to the container, as explained in Torizon Best Practices Guide, you have to enable the cgroup rule c 199:* rmw.
  • Namespace: To add the camera device to the container you can use bind mounts, but passing devices (using --device) is better for security since you avoid exposing a storage device that may be erased by an attacker.

To use camera devices, it's required to bind mount specific directories of the host machine. In other words, you have to guarantee access to certain directories. So, you have to use four bind mounts:

  • /dev: Mount /dev allows access to the devices that are attached to the local system.
  • /tmp: Mounting /tmp as a volume in the container allows your application to interact with Weston (GUI).
  • /sys: Grant access to kernel subsystems.
  • /var/run/dbus: Provides access to system services.

Another useful resource is displaying the camera output, especially during the setup and debugging phases of the application development, even if not necessary for the final application. To perform this, you can use a display by configuring a Weston-based container, or VNC. To read more about GUI in TorizonCore, refer to the section Graphical User Interface (GUI) in Torizon Best Practices Guide.

Dockerfile

The implementation details will be explained in this session. See the Quickstart Guide with the instructions about how to compile the image on a host pc and pull the image onto the board. You can also scp this file to the board and build it locally.

To build

danger

Make sure you have configured your Build Environment for Torizon Containers

Now it's a good time to use torizon-samples repository:

$ cd ~/torizon-samples/gstreamer/bash/simple-pipeline
docker build --build-arg BASE_NAME=wayland-base-vivante --build-arg IMAGE_ARCH=linux/arm64/v8 -t <your-dockerhub-username>/gst_example .

After the build, push the image to your Dockerhub account:

$ docker push <your-dockerhub-username>/gst_example

For more details about the dockerfile, refer to How to use Gstreamer on TorizonCore.

Hello world with a GStreamer pipeline

To run a GStreamer pipeline inside a container, in order to avoid possible problems in launching the containers that are required to start the video, make sure to stop all the other containers that might be running on your device.

# docker stop $(docker run -a -q)

Launching Weston Container

Considering that a streaming video application requires a GUI, you have to pull a Debian Bullseye container featuring the Weston Wayland compositor and start it on the module.

(Optional) pull the torizon/weston container image:

# docker pull torizon/weston:$CT_TAG_WESTON

Start the weston compositor:

# docker run -d --rm --name=weston --net=host --cap-add CAP_SYS_TTY_CONFIG \
-v /dev:/dev -v /tmp:/tmp -v /run/udev/:/run/udev/ \
--device-cgroup-rule='c 4:* rmw' --device-cgroup-rule='c 13:* rmw' \
--device-cgroup-rule='c 199:* rmw' --device-cgroup-rule='c 226:* rmw' \
torizon/weston:$CT_TAG_WESTON --developer weston-launch --tty=/dev/tty7 --user=torizon

Discovering the Video Capture Device

If it is the first time you are dealing with a specific video device and you're not sure which device is the capture one, you may follow the next steps to discover what is the right video capture device:

  1. Still outside the container, list the video devices by using ls /dev/video*.
# ls /dev/video*
/dev/video0 /dev/video1 /dev/video2 /dev/video3 /dev/video12 /dev/video13
  1. Start the previously built container by passing all the video devices under the flag --devices.
# docker run --rm -it -v /tmp:/tmp -v /var/run/dbus:/var/run/dbus -v /dev:/dev -v /sys:/sys \
--device /dev/video0 --device /dev/video1 --device /dev/video2 --device /dev/video3 --device /dev/video12 --device /dev/video13 \
--device-cgroup-rule='c 199:* rmw' \
<your-dockerhub-username>/<Dockerfile-name>
  1. Once inside the container, use the command v4l2-ctl --list-devices. Now you are able to see which devices are video capture ones.
## v4l2-ctl --list-devices
vpu B0 (platform:):
/dev/video12
/dev/video13

mxc-jpeg decoder (platform:58400000.jpegdec):
/dev/video0

mxc-jpeg decoder (platform:58450000.jpegenc):
/dev/video1

Video Capture 5 (usb-xhci-cdns3-1.2):
/dev/video2
/dev/video3
/dev/media0

  1. Use v4l2-ctl -D to find information about the Video Capture devices listed before. In this case, just the /dev/video2 and /dev/video3. As you are able to see, the camera is represented by /dev/video3.
## v4l2-ctl --device /dev/video3 -D
Driver Info:
Driver name : uvcvideo
Card type : Metadata 5
Bus info : usb-xhci-cdns3-1.2
Driver version : 5.4.161
Capabilities : 0x84a00001
Video Capture
Metadata Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04a00000
Metadata Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : uvcvideo
Model : USB 2.0 Camera: USB Camera
Serial : 01.00.00
Bus info : usb-xhci-cdns3-1.2
Media version : 5.4.161
Hardware revision: 0x00000003 (3)
Driver version : 5.4.161
Interface Info:
ID : 0x03000005
Type : V4L Video
Entity Info:
ID : 0x00000004 (4)
Name : Metadata 5
Function : V4L2 I/O
  1. With the right video device, from now on, when you start the container you just need to enable access for the specific camera device (in this case /dev/video3).

Launching the Gstreamer and Video4Linux2 Container

The next step is to launch the container with Gstreamer and Video4Linux2, making sure to pass the right video device /dev/video* under the parameter --device.

# docker run --rm -it -v /tmp:/tmp -v /var/run/dbus:/var/run/dbus -v /dev:/dev -v /sys:/sys \
--device /dev/<video-device> \
--device-cgroup-rule='c 199:* rmw' \
<your-dockerhub-username>/<Dockerfile-name>

Creating a Gstreamer pipeline

Once inside the container, you are able to use Video4Linux2 and Gstreamer resources. The basic structure of the pipeline relies on the usage of a data source (in this case a Video4Linux2 source), a filter, and a data sink, in this case, the video sink Wayland.

Gstreamer pipeline diagram

One of the simplest pipelines for showing video can be done the following way:

## gst-launch-1.0 <videosrc> ! <capsfilter> ! <videosink>

Or more specifically

## gst-launch-1.0 v4l2src device=</dev/video*> ! <capsfilter> ! fpsdisplaysink video-sink=waylandsink

If you want to learn more about how to elaborate a more complex Gstreamer pipeline, refer to How to use Gstreamer on TorizonCore.

To discover more properties of v4l2src to configure the pads, you can use gst-inspect:

gst-inspect output
gst-inspect-1.0 v4l2src

Factory Details:
Rank primary (256)
Long-name Video (video4linux2) Source
Klass Source/Video
Description Reads frames from a Video4Linux2 device
Author Edgard Lima <edgard.lima@gmail.com>, Stefan Kost <ensonic@users.sf.net>

Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/gstreamer-1.0/libgstvideo4linux2.so
Version 1.16.2
License LGPL
Source module gst-plugins-good
Binary package GStreamer Good Plug-ins source release
Origin URL Unknown package origin

GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseSrc
+----GstPushSrc
+----GstV4l2Src

Implemented Interfaces:
GstURIHandler
GstTuner
GstColorBalance
GstVideoOrientation

Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
image/jpeg
video/mpeg
mpegversion: 4
systemstream: false
video/mpeg
mpegversion: { (int)1, (int)2 }
video/mpegts
systemstream: true
video/x-bayer
format: { (string)bggr, (string)gbrg, (string)grbg, (string)rggb }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-cavs
video/x-divx
divxversion: [ 3, 6 ]
video/x-dv
systemstream: true
video/x-flash-video
flvversion: 1
video/x-fwht
video/x-h263
variant: itu
video/x-h264
stream-format: { (string)byte-stream, (string)avc }
alignment: au
video/x-h265
stream-format: byte-stream
alignment: au
video/x-pn-realvideo
video/x-pwc1
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-pwc2
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)RGB16, (string)BGR, (string)RGB, (string)RGBA, (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)YVU9, (string)YV12, (string)YUY2, (string)YVYU, (string)UYVY, (string)Y42B, (string)Y41B, (string)v308, (string)YUV9, (string)NV12_10LE, (string)NV12_64Z32, (string)NV24, (string)NV61, (string)NV16, (string)NV21, (string)NV12, (string)I420, (string)xRGB, (string)BGRA, (string)BGRx, (string)ARGB, (string)BGR15, (string)RGB15 }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-sonix
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-vp6-flash
video/x-vp8
video/x-vp9
video/x-wmv
wmvversion: 3
video/x-xvid

Element has no clocking capabilities.

URI handling capabilities:
Element can act as source.
Supported URI protocols:
v4l2

Pads:
SRC: 'src'
Pad Template: 'src'

Element Properties:
blocksize : Size in bytes to read per buffer (-1 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
brightness : Picture brightness, or more precisely, the black level
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
contrast : Picture contrast or luma gain
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
device : Device location
flags: readable, writable
String. Default: "/dev/video0"
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
device-name : Name of the device
flags: readable
String. Default: null
do-timestamp : Apply current stream time to buffers
flags: readable, writable
Boolean. Default: false
extra-controls : Extra v4l2 controls (CIDs) for the device
flags: readable, writable
Boxed pointer of type "GstStructure"
flags : Device type flags
flags: readable
Flags "GstV4l2DeviceTypeFlags" Default: 0x00000000, "(none)"
(0x00000001): capture - Device supports video capture
(0x00000002): output - Device supports video playback
(0x00000004): overlay - Device supports video overlay
(0x00000010): vbi-capture - Device supports the VBI capture
(0x00000020): vbi-output - Device supports the VBI output
(0x00010000): tuner - Device has a tuner or modulator
(0x00020000): audio - Device has audio inputs or outputs
force-aspect-ratio : When enabled, the pixel aspect ratio will be enforced
flags: readable, writable
Boolean. Default: true
hue : Hue or color balance
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
io-mode : I/O mode
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
name : The name of the object
flags: readable, writable
String. Default: "v4l2src0"
norm : video standard
flags: readable, writable
Enum "V4L2_TV_norms" Default: 0, "none"
(0): none - none
(45056): NTSC - NTSC
(4096): NTSC-M - NTSC-M
(8192): NTSC-M-JP - NTSC-M-JP
(32768): NTSC-M-KR - NTSC-M-KR
(16384): NTSC-443 - NTSC-443
(255): PAL - PAL
(7): PAL-BG - PAL-BG
(1): PAL-B - PAL-B
(2): PAL-B1 - PAL-B1
(4): PAL-G - PAL-G
(8): PAL-H - PAL-H
(16): PAL-I - PAL-I
(224): PAL-DK - PAL-DK
(32): PAL-D - PAL-D
(64): PAL-D1 - PAL-D1
(128): PAL-K - PAL-K
(256): PAL-M - PAL-M
(512): PAL-N - PAL-N
(1024): PAL-Nc - PAL-Nc
(2048): PAL-60 - PAL-60
(16711680): SECAM - SECAM
(65536): SECAM-B - SECAM-B
(262144): SECAM-G - SECAM-G
(524288): SECAM-H - SECAM-H
(3276800): SECAM-DK - SECAM-DK
(131072): SECAM-D - SECAM-D
(1048576): SECAM-K - SECAM-K
(2097152): SECAM-K1 - SECAM-K1
(4194304): SECAM-L - SECAM-L
(8388608): SECAM-Lc - SECAM-Lc
num-buffers : Number of buffers to output before sending EOS (-1 = unlimited)
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
pixel-aspect-ratio : Overwrite the pixel aspect ratio of the device
flags: readable, writable
String. Default: null
saturation : Picture color saturation or chroma gain
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
typefind : Run typefind before negotiating (deprecated, non-functional)
flags: readable, writable, deprecated
Boolean. Default: false

Element Signals:
"prepare-format" : void user_function (GstElement* object,
gint arg0,
GstCaps* arg1,
gpointer user_data);

As you are dealing with a raw video, you will have to use, in this case, the video/x-raw and configure its properties: format, width, height, and framerate.

## gst-launch-1.0 v4l2src device='/dev/<video-device>'  ! "video/x-raw, format=<video-format>, framerate=<framerate>, width=<supported-width>, height=<supported-height>" ! fpsdisplaysink video-sink=waylandsink

You also can check information about the sink using the gst-inspect:

gst-inspect output
# gst-inspect-1.0 waylandsink
Factory Details:
Rank primary + 2 (258)
Long-name wayland video sink
Klass Sink/Video
Description Output to wayland surface
Author Sreerenj Balachandran <sreerenj.balachandran@intel.com>, George Kiagiadakis <george.kiagiadakis@collabora.com>

Plugin Details:
Name waylandsink
Description Wayland Video Sink
Filename /usr/lib/gstreamer-1.0/libgstwaylandsink.so
Version 1.16.2
License LGPL
Source module gst-plugins-bad
Binary package GStreamer Bad Plug-ins source release
Origin URL Unknown package origin

GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseSink
+----GstVideoSink
+----GstWaylandSink

Implemented Interfaces:
GstVideoOverlay
GstWaylandVideo

Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)BGRx, (string)BGRA, (string)RGBx, (string)xBGR, (string)xRGB, (string)RGBA, (string)ABGR, (string)ARGB, (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)YUY2, (string)YVYU, (string)UYVY, (string)AYUV, (string)NV12, (string)NV21, (string)NV16, (string)YUV9, (string)YVU9, (string)Y41B, (string)I420, (string)YV12, (string)Y42B, (string)v308 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw(memory:DMABuf)
format: { (string)BGRx, (string)BGRA, (string)RGBx, (string)xBGR, (string)xRGB, (string)RGBA, (string)ABGR, (string)ARGB, (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)YUY2, (string)YVYU, (string)UYVY, (string)AYUV, (string)NV12, (string)NV21, (string)NV16, (string)YUV9, (string)YVU9, (string)Y41B, (string)I420, (string)YV12, (string)Y42B, (string)v308 }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
SINK: 'sink'
Pad Template: 'sink'

Element Properties:
alpha : Wayland surface alpha value, apply custom alpha value to wayland surface
flags: readable, writable
Float. Range: 0 - 1 Default: 0
async : Go asynchronously to PAUSED
flags: readable, writable
Boolean. Default: true
blocksize : Size in bytes to pull per buffer (0 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
display : Wayland display name to connect to, if not supplied via the GstContext
flags: readable, writable
String. Default: null
enable-last-sample : Enable the last-sample property
flags: readable, writable
Boolean. Default: true
enable-tile : When enabled, the sink propose VSI tile modifier to VPU
flags: readable, writable
Boolean. Default: false
fullscreen : Whether the surface should be made fullscreen
flags: readable, writable
Boolean. Default: false
last-sample : The last sample received in the sink
flags: readable
Boxed pointer of type "GstSample"
max-bitrate : The maximum bits per second to render (0 = disabled)
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
max-lateness : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
flags: readable, writable
Integer64. Range: -1 - 9223372036854775807 Default: 5000000
name : The name of the object
flags: readable, writable
String. Default: "waylandsink0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
processing-deadline : Maximum processing deadline in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 15000000
qos : Generate Quality-of-Service events upstream
flags: readable, writable
Boolean. Default: true
render-delay : Additional render delay of the sink in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
show-preroll-frame : Whether to render video frames during preroll
flags: readable, writable
Boolean. Default: true
sync : Sync on the clock
flags: readable, writable
Boolean. Default: true
throttle-time : The time to keep between rendered buffers (0 = disabled)
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
ts-offset : Timestamp offset in nanoseconds
flags: readable, writable
Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
window-height : Wayland sink preferred window height in pixel
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
window-width : Wayland sink preferred window width in pixel
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
info

You have to keep in mind that different cameras may support different video formats, resolutions (height and width), etc. Then, even the simplest pipeline can vary

So, the final structure of the pipeline should be similar to the following one:

## gst-launch-1.0 v4l2src device='/dev/<video-device>'  ! "video/x-raw, format=<video-format>, framerate=<framerate>, width=<supported-width>, height=<supported-height>" ! fpsdisplaysink video-sink=waylandsink text-overlay=<true-or-false> sync=<true-or-false>

The following subsections cover the specific considerations of the described process to use MIPI CSI-2 cameras and Webcams.

USB Camera

The USB 2.0 Host interface is available on all the modules to connect USB Cameras. However, USB 3.0 is only available on specific modules from the Apalis SoM Family and the Verdin SoM Family, check the datasheets for details. Our article Webcam (Linux) is a starting point on how to set up a USB camera in Linux, as mentioned in Camera on Toradex Computer on Modules.

The following pipeline was used to show video from a webcam:

## gst-launch-1.0 v4l2src device='/dev/video3'  ! "video/x-raw, format=YUY2, framerate=5/1, width=640, height=480" ! fpsdisplaysink video-sink=waylandsink text-overlay=false sync=false

MIPI CSI-2 Camera

To use a MIPI-CSI Camera you have to make sure that your module and carrier board have a MIPI-CSI connection. It’s important to highlight the following considerations from Camera on Toradex Computer on Modules:

  • The Apalis module family provides MIPI CSI-2 interfaces on type-specific pins.
  • The Colibri module family does not provide MIPI-CSI interfaces. Some SoMs, such as the Colibri iMX8X, have a MIPI CSI-2 interface exposed in an additional FFC connector.
  • The Verdin module family provides one quad-lane MIPI CSI-2 interface on reserved pins, according to the Verdin Family Specification. That means that, as long as the SoC used in a Verdin SoM has MIPI CSI-2, we make sure it's always exposed on fixed pins.

Depending on the MIPI CSI-2 Camera you are using, you may enable a specific device tree overlay. As in this example, it was used the Camera Set 5MP AR0521 Color, the overlay process using TorizonCore Builder was described at First Steps with CSI Camera Set 5MP AR0521 Color (Torizon).

With the overlay enabled, the next step is to launch a video stream using a pipeline. To achieve this, you should follow a process similar to the one described in the article First Steps with CSI Camera Set 5MP AR0521 Color (Torizon).

The following pipeline was used to show video with Camera Set 5MP AR0521 Color:

## gst-launch-1.0 v4l2src device='/dev/video0'  ! "video/x-raw, format=RGB16, framerate=30/1, width=1920, height=1080" ! fpsdisplaysink video-sink=waylandsink text-overlay=false sync=false

Hello world with Python and OpenCV

This hello-world application, like the usage of Gstreamer directly on the terminal, is just a video stream. For this application you are going to use OpenCV together with V4L2 and GStreamer, using Visual Studio Code Extension for Torizon. If you want to learn more about OpenCV, refer to Torizon Sample: Using OpenCV for Computer Vision.

So, the first step is to create a python application with QML, as described at Python development on TorizonCore. We’re not going to use QML, we just need the GUI application launching Weston.

Then, go to Torizon Extension and proceed with the described modifications. Add the following volumes under the parameter volumes:

  • key = /dev, value=/dev
  • key = /tmp, value=/tmp
  • Key = /sys, value=/sys
  • key = /var/run/dbus, value=/var/run/dbus

Add access to the camera device (/dev/video2) under the parameter devices. Add a a cgroup rule under the extraparms parameter as follows: key=device_cgroup_rules and value=[ "c 199:* rmw" ]

Finally, modify the field extrapackages including all the packages and plugins explained before, and add the OpenCV package for python3, which is python3-opencv.

python3-opencv v4l-utils gstreamer1.0-qt5 libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-pulseaudio

The code must be written inside the main.py and begin with the import of the OpenCV library, called cv2

import cv2

OpenCV provides the VideoCapture class used to capture the video. A pipeline similar to the one created before is going to be the input parameter of the class. The difference between the pipelines is the use of appsink instead of waylandsink, as the pipeline will send data to your program, not to a display.

cap = cv2.VideoCapture("v4l2src device=/dev/video2 ! video/x-raw, width=640, height=480, format=YUY2 ! videoconvert ! video/x-raw, format=BGR ! appsink")

To capture and read the frames we’re going to use read:

ret, frame = cap.read() # Capture frame

And to display the frames we can use imshow:

cv2.imshow('frame', frame) # Display the frames

Then, release the capture and destroy the window:

cap.release() # Release the capture object
cv2.destroyAllWindows() # Destroy all the windows
tip

To read frames and display them continuously, you can put the capture and display lines inside a loop.

Send Feedback!