First Steps with Vision Components MIPI Camera Modules (Torizon)
Introduction
This article describes how to start working with Vision Components VC MIPI CSI-2 camera modules on Torizon OS.
The VC MIPI modules supported in this article share common software components, such as the camera driver and device tree overlay. Therefore, you can use Vision Components Yocto Project layers to build images for all supported camera modules.
The scope of this article is specific to the VC MIPI modules. For general information about cameras on Torizon OS, see How to use Cameras on Torizon.
Main Features
This article covers the following Vision Components MIPI modules:
IMX327C
Camera Module Features:
- Sensor: IMX327 from Sony® (improved version of the IMX290)
- Variant: Color (C)
- Sensor Resolution and Optical Format: 2MP CMOS Image Sensor, 1/2.8" optical format
- Native Resolution: 1920×1080 (Full HD)
- Sensor Type: CMOS Electronic Rolling Shutter Sensor
- Interface: MIPI CSI-2
Sensor Features:
- Pixel size: 2.9μm × 2.9μm
- Shutter type: Rolling Shutter
- Max frame rate: 60 fps at 10-bit (full resolution)
The IMX327 is a low-light optimized sensor, which makes it a common choice for surveillance, automotive, and machine-vision applications operating in poor lighting conditions.
Hardware Documentation
VC MIPI IMX327C Hardware Operating Manual
Compatible Products
Hardware
The following Toradex carrier boards that support the MIPI CSI-2 interface connect to the camera kits out-of-the-box, hardware-wise:
This camera module is supported for the following SoMs:
Software
This workflow requires Torizon OS 7.7.0 or newer. The kernel patches that the VC MIPI capture pipeline depends on are part of the stock Torizon OS kernel from that release onward, so you do not need a custom kernel. Earlier releases do not carry these patches.
To integrate the camera, you add the driver and the overlay to a stock image and then build the ISP container, as shown in Software Setup.
Hardware Setup
What I Need to Order
The VC MIPI modules are part of a modular system. Hence, you can order each part (camera module, connector cable and lens) separately.
See the compatible products.
The following list describes all components that you should order:
-
Camera Module — Currently this guide documents the following VC MIPI camera module:
- VC MIPI IMX327C (2MP/1080p color optical sensor module, Sony® STARVIS™).
-
FPC Cable — The FPC cable connects the camera module to the carrier board and must match your processor board.
- 22-pin FPC cable (module side), in the length and board variant suitable for your carrier board.
-
Lens and Lens Holder — The module is shipped without optics. A lens and a matching lens holder are ordered separately.
- Lens holder for S-mount, C-mount, or CS-mount lenses.
- Lens matching the chosen holder and your application.
Where Do I Order
You can order the Toradex computer-on-modules and carrier boards online in the Toradex Webshop.
You can order the Vision Components camera module and accessories by directly contacting Vision Components.
Cable Connection
Connect the flat cable to the camera module, making sure the correct side of the cable faces the module and the board, as shown in the following pictures:
Please disconnect any power and USB cables from the board before connecting the flat cable to the MIPI CSI-2 bus to avoid any damage to the camera!
Connection to Dahlia Carrier Board
- For the Dahlia Carrier Board, the MIPI-CSI camera interface is on connector X16.
Connection to Mallow Carrier Board
- For the Mallow Carrier Board, the MIPI-CSI camera interface is on connector X3.
Connection to Verdin Development Board
- For the Verdin Development Board, the MIPI-CSI camera interface is on connector X47.
The flat ribbon cable used to connect the carrier board and the camera adapter is fragile. If your camera does not work, make sure to test the cable connections.
Software Setup
Prerequisites
Before you start, make sure you have:
- TorizonCore Builder and Docker installed.
- A Docker Hub account, to store the ISP container image.
- A build environment configured for Torizon containers, including ARM emulation.
- A GStreamer container image for video streaming. You can build one by following How to use GStreamer on Torizon OS.
First, create a folder in which TorizonCore Builder should access the host-side files:
$ mkdir -p ~/vc-torizon && cd ~/vc-torizon
Throughout this article, we refer to ~/vc-torizon as the working directory. If you use a different directory, remember to adjust all references to the working directory accordingly.
Adding the Kernel Module and the Device Tree Overlay
In this step you use TorizonCore Builder to produce a custom Torizon OS image that contains the VC MIPI driver and the device tree overlay.
This procedure requires Torizon OS 7.7.0 or later. Earlier releases are not supported.
Step 1 — Prepare the kernel module sources
The source code for building the VC MIPI driver belongs to two different Vision Components repositories. Use the commands below to download the source files from the meta-vc-mipiat the v0.6.0 state. On those versions, you can compile the driver as-is against the Torizon OS kernel. You should not use default branches, since they move over time and may not build.
$ mkdir -p vc-mipi-module
$ wget -P vc-mipi-module https://raw.githubusercontent.com/VC-MIPI-modules/vc_mipi_nxp/2576768b52973be4105b67010350ee57e2643a4b/src/vc_mipi_camera.c
$ wget -P vc-mipi-module https://raw.githubusercontent.com/VC-MIPI-modules/vc_mipi_core/eb686ed72a4e78966ba68954fd863a50d172d654/vc_mipi_core.c
$ wget -P vc-mipi-module https://raw.githubusercontent.com/VC-MIPI-modules/vc_mipi_core/eb686ed72a4e78966ba68954fd863a50d172d654/vc_mipi_core.h
$ wget -P vc-mipi-module https://raw.githubusercontent.com/VC-MIPI-modules/vc_mipi_core/eb686ed72a4e78966ba68954fd863a50d172d654/vc_mipi_modules.c
$ wget -P vc-mipi-module https://raw.githubusercontent.com/VC-MIPI-modules/vc_mipi_core/eb686ed72a4e78966ba68954fd863a50d172d654/vc_mipi_modules.h
The driver also needs the vvsensor.h header from the NXP isp-imx package:
$ cd /tmp
$ wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/isp-imx-4.2.2.24.4-8527c7b.bin
$ chmod +x isp-imx-4.2.2.24.4-8527c7b.bin
$ ./isp-imx-4.2.2.24.4-8527c7b.bin --auto-accept
$ cp /tmp/isp-imx-4.2.2.24.4-8527c7b/vvcam/common/vvsensor.h ~/vc-torizon/vc-mipi-module/
Create the file ~/vc-torizon/vc-mipi-module/Makefile with the following content:
obj-m := vc-mipi-vvcam.o
vc-mipi-vvcam-objs := vc_mipi_camera.o vc_mipi_core.o vc_mipi_modules.o
EXTRA_CFLAGS += -DENABLE_VVCAM
SRC := $(shell pwd)
all:
$(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules
clean:
$(MAKE) -C $(KERNEL_SRC) M=$(SRC) clean
Indent the Makefile recipe lines with tab characters, not spaces. Copying from a browser often replaces them.
Verify the contents of the vc-mipi-module/ directory:
$ ls -1 vc-mipi-module/
The output should include the following files:
Makefile
vc_mipi_camera.c
vc_mipi_core.c
vc_mipi_core.h
vc_mipi_modules.c
vc_mipi_modules.h
vvsensor.h
Step 2 — Prepare the device tree overlay
Vision Components provides a single overlay that enables both capture paths, ISI (Image Sensing Interface) and ISP. Download it:
$ cd ~/vc-torizon
$ mkdir -p overlays
$ wget -O overlays/verdin-imx8mp_vc_mipi_overlay.dts \
https://raw.githubusercontent.com/VC-MIPI-modules/meta-vc-mipi/9e9aa2a57d7751514197f380df6e256485ff73c7/recipes-kernel/linux/device-tree-overlays/verdin-imx8mp_vc_mipi_overlay.dts
The overlay needs the Toradex kernel dt-bindings headers to compile:
$ git clone https://git.toradex.com/linux-toradex.git \
--branch toradex_6.6-2.2.x-imx --depth 3 /tmp/lt-headers
$ mkdir -p ~/vc-torizon/include
$ cp -r /tmp/lt-headers/include/dt-bindings ~/vc-torizon/include/
The toradex_6.6-2.2.x-imx branch matches the kernel used by Torizon OS 7.7.0 on the Verdin iMX8M Plus. If you use a newer Torizon OS release, check the corresponding kernel version in the Embedded Linux Release Matrix and use the matching branch from the Toradex Linux kernel repository.
Step 3 — Get a base Torizon OS image
TorizonCore Builder requires a base Torizon OS image to build the customized image. You can either download the image manually or configure TorizonCore Builder to fetch it from the Toradex feed.
To use a local image, download a Torizon OS image (version 7.7.0 or newer) for verdin-imx8mp from the Toradex Download Links page and place the .tar file in the working directory. The downloaded image will be used as an input for TorizonCore Builder.
Alternatively, you can skip the download and let TorizonCore Builder fetch the base image.
The next step shows how to configure tcbuild.yaml for either option.
Step 4 — Create the tcbuild.yaml file
Create ~/vc-torizon/tcbuild.yaml with the content below. The example uses a local base image, so replace the local value in the input section with the exact filename of the .tar image you downloaded in the previous step.
To let TorizonCore Builder fetch the base image instead, replace the local entry with a toradex-feed configuration.
input:
easy-installer:
local: "torizon-docker-verdin-imx8mp-Tezi_7.7.0+build.40.tar"
customization:
kernel:
modules:
- source-dir: vc-mipi-module/
autoload: yes
device-tree:
include-dirs:
- include/
overlays:
add:
- overlays/verdin-imx8mp_vc_mipi_overlay.dts
output:
easy-installer:
local: torizon-vc-Tezi
Step 5 — Build the custom image
$ cd ~/vc-torizon
$ torizoncore-builder build
TorizonCore Builder compiles the overlay against the stock device tree and the module against the base kernel, then generates the torizon-vc-Tezi directory.
Warnings from vendor .dtsi files during overlay compilation are expected and do not indicate a failure.
Step 6 — Deploy the image
Install the resulting image on the board with the Toradex Easy Installer, through SSH, or via Torizon Cloud. For the available options, see Deploy The Custom Toradex Easy Installer Image.
Both commands must return output. If either is empty, review the tcbuild.yaml file and rebuild the image.
Step 7 — Verify the module and the overlay
After the board boots, connect to it via SSH or a serial terminal and confirm that the device tree overlay is loaded:
# ls /proc/device-tree/chosen/overlays/ | grep vc_mipi
verdin-imx8mp_vc_mipi_overlay.dtbo
Then, confirm that the VC MIPI kernel module is loaded:
# lsmod | grep vc_mipi
vc_mipi_vvcam 94208 1
If either command returns no output, review the tcbuild.yaml file and rebuild the image.
Building the ISP Container
The ISP container is the userspace side of the camera pipeline. It detects the connected sensor, generates the matching tuning files, loads the VeriSilicon ISP modules, and starts isp_media_server. The server performs demosaicing, Automatic White Balance (AWB), color correction, black level correction, and dewarp, and outputs a YUV stream on a dedicated capture device.
Step 1 — Download the Vision Components patches
The container builds the NXP isp-imx package with three Vision Components patches applied:
$ mkdir -p ~/vc-torizon/isp-container/patches
$ cd ~/vc-torizon/isp-container/patches
$ base="https://raw.githubusercontent.com/VC-MIPI-modules/meta-vc-mipi/9e9aa2a57d7751514197f380df6e256485ff73c7/recipes-bsp/isp-imx/isp-imx"
$ wget "$base/0001-Added-VC-MIPI-CSI-2-driver.patch"
$ wget "$base/0001-Improve-default-settings-in-vc_mipi_dewarp_template..patch"
$ wget "$base/0001-Fix-vc-mipi-setup.sh-to-be-more-generic-and-usable-o.patch"
Step 2 — Create the Dockerfile
Create ~/vc-torizon/isp-container/Dockerfile with the following content:
ARG BASE_NAME=wayland-base-imx8
ARG IMAGE_ARCH=linux/arm64
ARG IMAGE_TAG=4.15.0
ARG DOCKER_REGISTRY=torizon
FROM ubuntu:24.04 AS newer-cxx-runtime
RUN apt-get update && apt-get install -y libstdc++6 \
&& rm -rf /var/lib/apt/lists/*
FROM $DOCKER_REGISTRY/$BASE_NAME:$IMAGE_TAG AS vc-mipi-build
ARG ISP_IMX_VERSION="4.2.2.24.4"
ARG ISP_IMX_SRCREV="8527c7b"
ARG ISP_IMX_URL="https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/isp-imx-${ISP_IMX_VERSION}-${ISP_IMX_SRCREV}.bin"
RUN apt-get update && apt-get install -y \
cmake build-essential gcc g++ git wget \
libboost-all-dev libdrm-dev \
libtinyxml2-dev libjsoncpp-dev \
patchelf patch kmod \
&& rm -rf /var/lib/apt/lists/*
RUN ln -sf /usr/lib/aarch64-linux-gnu/libtinyxml2.so /usr/lib/libtinyxml2.so 2>/dev/null || true
RUN ln -sf /usr/include/libdrm /usr/include/drm 2>/dev/null || true
WORKDIR /downloads
RUN wget -q -O isp-imx-${ISP_IMX_VERSION}-${ISP_IMX_SRCREV}.bin ${ISP_IMX_URL} && \
chmod +x isp-imx-${ISP_IMX_VERSION}-${ISP_IMX_SRCREV}.bin && \
./isp-imx-${ISP_IMX_VERSION}-${ISP_IMX_SRCREV}.bin --auto-accept --force
WORKDIR /downloads/isp-imx-${ISP_IMX_VERSION}-${ISP_IMX_SRCREV}
COPY patches/ /downloads/patches/
RUN git apply /downloads/patches/0001-Added-VC-MIPI-CSI-2-driver.patch && \
git apply /downloads/patches/0001-Improve-default-settings-in-vc_mipi_dewarp_template..patch && \
git apply /downloads/patches/0001-Fix-vc-mipi-setup.sh-to-be-more-generic-and-usable-o.patch
RUN ./build-all-isp.sh Release v4l2
RUN cmake --build appshell/build --target vc-mipi.drv
RUN mkdir -p /isp-install/bin /isp-install/lib && \
cp mediacontrol/install/bin/isp_media_server /isp-install/bin/ && \
cp -r mediacontrol/install/lib/* /isp-install/lib/ && \
cp -r appshell/shell_libs/ispcore/ARM64/* /isp-install/lib/ && \
cp -r dewarp/proprietories/hal/lib/* /isp-install/lib/ && \
( cp -r build_output_release_v4l2/usr/lib/* /isp-install/lib/ 2>/dev/null || true ) && \
cp /usr/lib/aarch64-linux-gnu/libjsoncpp.so* /isp-install/lib/ && \
cp /usr/lib/aarch64-linux-gnu/libstdc++.so.6* /isp-install/lib/ && \
find /usr/lib /lib -name "libtinyxml2.so*" -exec cp -av {} /isp-install/lib/ \; && \
cp appshell/build/generated/release/bin/vc-mipi.drv /isp-install/bin/ && \
cp build_output_release_v4l2/opt/imx8-isp/bin/run.sh /isp-install/bin/ && \
cp build_output_release_v4l2/opt/imx8-isp/bin/start_isp.sh /isp-install/bin/ && \
cp build_output_release_v4l2/opt/imx8-isp/bin/vc-mipi-setup.sh /isp-install/bin/ && \
cp build_output_release_v4l2/opt/imx8-isp/bin/vc_mipi_tuning_template.xml /isp-install/bin/ && \
cp -r build_output_release_v4l2/opt/imx8-isp/bin/dewarp_config /isp-install/bin/
FROM $DOCKER_REGISTRY/$BASE_NAME:$IMAGE_TAG AS vc-mipi-isp
RUN apt-get update && apt-get install -y \
kmod procps i2c-tools v4l-utils libstdc++6 \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir -p /opt/imx8-isp/bin/dewarp_config
COPY /isp-install/bin/ /opt/imx8-isp/bin/
COPY /isp-install/lib/ /usr/lib/
COPY /usr/lib/aarch64-linux-gnu/libstdc++.so.6* /usr/lib/
ENV LD_LIBRARY_PATH=/usr/lib:/opt/imx8-isp/lib
RUN cd /usr/lib && \
if [ ! -e libtinyxml2.so.10 ] && [ -e libtinyxml2.so.9 ]; then \
ln -s libtinyxml2.so.9 libtinyxml2.so.10; \
fi
RUN chmod +x /opt/imx8-isp/bin/run.sh \
/opt/imx8-isp/bin/start_isp.sh \
/opt/imx8-isp/bin/vc-mipi-setup.sh \
/opt/imx8-isp/bin/isp_media_server
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
Step 3 — Create the entrypoint script
Create ~/vc-torizon/isp-container/entrypoint.sh with the following content. The script prepares a writable copy of the ISP files, runs the camera detection, starts the ISP server, and sets the sensor format on the media graph.
#!/bin/bash
set -e
ISP_DIR="/opt/imx8-isp/bin"
ISP_VAR="/var/imx8-isp/bin"
# 1. Copy to /var (writable)
mkdir -p ${ISP_VAR}/dewarp_config
cp -rf ${ISP_DIR}/* ${ISP_VAR}/ 2>/dev/null || true
chmod -R 755 ${ISP_VAR}
# 1.5 Create symlink /opt/imx8-isp/bin -> /var/imx8-isp/bin
mkdir -p /opt/imx8-isp
rm -rf /opt/imx8-isp/bin
ln -s ${ISP_VAR} /opt/imx8-isp/bin
# 2. Fix the print() in vc-mipi-setup.sh
sed -i 's/echo \${1} >$(tty)/echo \${1} >\&2/' ${ISP_VAR}/vc-mipi-setup.sh 2>/dev/null || true
# 3. Ensure bash in the scripts
sed -i 's|#!/bin/sh|#!/bin/bash|' ${ISP_VAR}/run.sh
sed -i 's|#!/bin/sh|#!/bin/bash|' ${ISP_VAR}/start_isp.sh
# 4. Remove an invalid call inside start_isp.sh (the correct setup is done below)
sed -i 's|^\./vc-mipi-setup.sh --service create|# disabled by container entrypoint: setup-c0 already done|' ${ISP_VAR}/start_isp.sh 2>/dev/null || true
# 5. Clean up old tuning files
rm -f ${ISP_VAR}/Sensor0_Entry.cfg
rm -f ${ISP_VAR}/Sensor1_Entry.cfg
rm -f ${ISP_VAR}/vc_imx327c_tuning.xml
rm -f ${ISP_VAR}/dewarp_config/vc_imx327c_dewarp.json
# 6. Run the camera setup
cd ${ISP_VAR}
./vc-mipi-setup.sh --service setup-c0 1920 1080
# 7. Start the ISP
bash ./start_isp.sh &
ISP_PID=$!
# 8. Wait for start_isp.sh to reload the modules and create the media graph.
# A fixed delay is unreliable: on the first boot the kernel modules take longer
# to load, so poll for the graph instead of sleeping for a fixed time.
echo "Waiting for the media graph to come up..."
graph_ready=0
for i in $(seq 30); do
if [ -e /dev/media0 ] && \
media-ctl -d /dev/media0 -p 2>/dev/null | grep -q "vc-mipi-cam"; then
graph_ready=1
break
fi
sleep 1
done
if [ "$graph_ready" -ne 1 ]; then
echo "WARNING: media graph with the vc-mipi-cam entity did not come up after 30 s." >&2
fi
# 9. Set the Bayer format reported by the sensor, retrying until it succeeds
# so a slow bring-up does not leave the format unset silently.
echo "Configuring VC-MIPI sensor format on /dev/media0..."
format_set=0
for i in $(seq 10); do
if media-ctl -d /dev/media0 \
--set-v4l2 '"vc-mipi-cam 2-001a":0 [fmt:SRGGB10_1X10/1920x1080]'; then
format_set=1
echo "Sensor format configured."
break
fi
echo "media-ctl not ready yet, retrying ($i/10)..."
sleep 1
done
if [ "$format_set" -ne 1 ]; then
echo "ERROR: could not set the sensor format on /dev/media0." >&2
fi
echo "VC-MIPI ISP container initialized."
wait $ISP_PID
The resolution and the I²C address in the script match the IMX327C at 1080p. Adjust them if you use a different module.
Step 4 — Build and push the container image
The container targets linux/arm64. Enable ARM emulation on the host before building:
$ docker run --rm -it --privileged torizon/binfmt
Then build the image and push it to your Docker Hub account, replacing <your-dockerhub-username> with your own:
$ docker login
$ docker build \
--platform linux/arm64 \
-t <your-dockerhub-username>/vc-mipi-isp:latest \
~/vc-torizon/isp-container/
$ docker push <your-dockerhub-username>/vc-mipi-isp:latest
After completing the Software Setup steps, the working directory should have a structure similar to the following:
├── vc-mipi-module/ # kernel module sources
├── overlays/ # device tree overlay source
├── include/ # dt-bindings headers
├── isp-container/ # ISP container files
├── torizon-vc-Tezi/ # custom image generated by TorizonCore Builder
└── tcbuild.yaml # TorizonCore Builder configuration file
The torizon-vc-Tezi directory name may vary depending on the output.easy-installer.local value configured in tcbuild.yaml.
Camera Usage
This section runs on the board. We recommend connecting a display to visualize the captured image during testing.
Launch the ISP Container
With the camera connected, start the ISP container:
# docker run -d --name=vc-mipi-isp --privileged \
-v /usr/lib/modules:/usr/lib/modules \
-v /var/imx8-isp:/var/imx8-isp \
-v /dev:/dev \
-v /sys:/sys \
<your-dockerhub-username>/vc-mipi-isp:latest
Check the logs to confirm that the camera was detected:
# docker logs vc-mipi-isp | tail -20
The output includes the detected sensor and the configured image size:
Found VC MIPI camera on i2c-2 (i2c@30a40000)
Detected device is a VC MIPI IMX327C camera
Image size will be set to 1920x1080
...
Starting isp_media_server with configuration file CAMERA0
If the log does not report a detected camera, the cause is usually the device tree overlay, which controls I²C and lane detection. Check the cable connection first. If the cable is seated correctly but the overlay or module is still missing (as shown by the commands in Step 7), correct the tcbuild.yaml file and repeat Steps 4–6 to rebuild and redeploy the image.
Launch the Weston Container
Weston provides the Wayland compositor used to display the video output.
If you are running an evaluation image, first stop the Easy Pairing container, which holds the display:
# docker update --restart=no torizon-easy-pairing-bash-1
# docker stop torizon-easy-pairing-bash-1
Then start Weston. For the Verdin iMX8M Plus:
# docker run -d --name=weston --net=host \
--cap-add CAP_SYS_TTY_CONFIG \
-v /dev:/dev -v /tmp:/tmp -v /run/udev/:/run/udev/ \
--device-cgroup-rule="c 4:* rmw" --device-cgroup-rule="c 253:* rmw" \
--device-cgroup-rule="c 13:* rmw" --device-cgroup-rule="c 226:* rmw" \
--device-cgroup-rule="c 10:223 rmw" --device-cgroup-rule="c 199:* rmw" \
torizon/weston-imx8:4 --developer
Confirm that Weston took over the display:
# ls -l /tmp/1000-runtime-dir/wayland-0
srwxr-xr-x 1 torizon torizon 0 <date> <time> /tmp/1000-runtime-dir/wayland-0
To learn more about the device-cgroup-rule options used above, refer to the Hardware Access through Control Group Rules (cgroup) section of the Torizon Best Practices Guide.
Identify the Capture Devices
The overlay enables two independent capture paths, which appear as separate video devices. The viv_v4l2_device node in the table below is the VeriSilicon ISP V4L2 driver, while the mxc-isi-* nodes belong to the i.MX ISI:
| Media device | Node | Driver | Role |
|---|---|---|---|
/dev/media0 | /dev/video3 | mxc-isi-cap | ISI raw Bayer capture |
/dev/media0 | /dev/video2 | mxc-isi-m2m | Memory-to-memory, not a capture device |
/dev/media1 | /dev/video4 | viv_v4l2_device | Processed ISP output |
The vc-mipi-cam sensor entity is exposed through the /dev/media0 media graph. Therefore, the entrypoint configures the sensor format on /dev/media0, while /dev/media1 exposes the processed ISP output.
The kernel assigns these node numbers at boot time and they can change between images or kernel versions. To list the devices on your board, run:
# docker exec vc-mipi-isp v4l2-ctl --list-devices
The ISI path delivers raw Bayer data with no demosaicing or white balance, so sending it directly to a display produces a green image. Use the ISP node for a ready-to-display stream.
Stream the Video with GStreamer
Start a GStreamer container. You need an image with GStreamer and the V4L2 and Wayland plugins. If you have not built one yet, follow How to use GStreamer on Torizon OS, then use its name in place of <your-gstreamer-image> below.
# docker run --rm -it --privileged --entrypoint '/bin/bash' \
-v /tmp:/tmp -v /var/run/dbus:/var/run/dbus -v /dev:/dev -v /sys:/sys \
-e XDG_RUNTIME_DIR=/tmp/1000-runtime-dir -e WAYLAND_DISPLAY=wayland-0 \
<your-gstreamer-image>
Inside the container, stream the ISP output. Replace /dev/video4 with the ISP node you confirmed in Identify the Capture Devices, since the number can change between images and kernel versions.
Even though the camera module has a lower native resolution, the ISP pipeline outputs a 4K frame, so the pipeline below scales it down to the display resolution:
## gst-launch-1.0 -v v4l2src device=/dev/video4 ! \
videoconvert ! videoscale ! \
video/x-raw,width=1920,height=1080 ! \
waylandsink sync=false
Do not set width and height on v4l2src. Doing so captures only the top-left quadrant of the 4K frame. Let v4l2src negotiate and use videoscale to resize.
V4L2 and GStreamer use different names for the same format: YUYV in V4L2 is YUY2 in GStreamer. Use format=YUY2 if you set the format explicitly in the caps.
To check the display path alone, without involving the camera, run:
## gst-launch-1.0 videotestsrc ! videoconvert ! waylandsink sync=false
If no color bars appear, the problem is in the display or in Weston, not in the camera.
Capture a Frame Without a Display
On a headless setup, capture a single frame from the ISP node you identified in the Identify the Capture Devices and copy it to your host:
## gst-launch-1.0 -v v4l2src device=/dev/video4 num-buffers=1 ! \
videoconvert ! jpegenc ! filesink location=/tmp/test.jpg
$ scp torizon@<board-ip>:/tmp/test.jpg .
Restart the Containers After a Reboot
The device tree overlay and the VC MIPI driver are part of the image and load automatically at every boot. The containers do not: a reboot stops them, but does not remove them.
Start the existing containers again with docker start, not docker run:
# docker start vc-mipi-isp
# docker start weston
The GStreamer container runs with --rm, so recreate it with docker run each time.
Start the Containers Automatically
To bring the camera up at every boot, describe the ISP and Weston containers in a docker-compose.yml file with restart: unless-stopped:
services:
vc-mipi-isp:
image: <your-dockerhub-username>/vc-mipi-isp:latest
privileged: true
volumes:
- /usr/lib/modules:/usr/lib/modules
- /var/imx8-isp:/var/imx8-isp
- /dev:/dev
- /sys:/sys
restart: unless-stopped
weston:
image: torizon/weston-imx8:4
command: --developer
network_mode: host
cap_add: [CAP_SYS_TTY_CONFIG]
volumes:
- /dev:/dev
- /tmp:/tmp
- /run/udev/:/run/udev/
device_cgroup_rules:
- "c 4:* rmw"
- "c 253:* rmw"
- "c 13:* rmw"
- "c 226:* rmw"
- "c 10:223 rmw"
- "c 199:* rmw"
restart: unless-stopped
Start the services from the directory containing the file:
# docker compose up -d
Only the viewing container then remains to be started manually.
How to Use Video Streams on Linux — Video4Linux & GStreamer
For more details about using Video4Linux and GStreamer tools to interact with cameras and collect video frames, see Cameras on Toradex System on Modules and How to use Gstreamer on Torizon OS.
Limitations
Sensors wider than 4K. Sensors that output images wider than 4096 pixels require an additional kernel patch that is not part of the stock Torizon OS kernel and that TorizonCore Builder cannot apply, because it patches kernel source. These sensors currently require a Yocto Project build. Sensors up to 4K wide, including the IMX327C, work with the workflow described in this article.
Color tuning. The container generates a default tuning file for the detected sensor. It is a functional starting point rather than a calibrated profile, and the output may show a slight color cast. Adjusting it properly requires a reference color chart under controlled illumination. To start from the current tuning file, extract it from the container:
# docker exec vc-mipi-isp cat /var/imx8-isp/bin/vc_imx327c_tuning.xml > current_tuning.xml
Having Trouble?
Please contact our technical support. Various options for technical support are mentioned in the article below.