Search by Tags

Audio/Video over RTP With GStreamer (Linux)

 

Article updated at 17 Aug 2021
Compare with Revision



Subscribe for this article updates

Introduction

Real-time Transport Protocol (RTP) is a very common network protocol for delivering media over IP networks.

On the board, you will need a GStreamer pipeline that encodes the raw video, adds the RTP payload, and sends over a network sink. A generic pipeline would look as follows:

video source ! video encoder ! RTP payload ! network sink
  • Video source: often it is a camera, but it can be a video from a file or a test pattern, for example.
  • Video encoder: a video encoder as H.264, VP8, JPEG and others.
  • RTP payload: an RTP payload that matches the video encoder.
  • Network sink: a video sync that streams over the network, often via UDP.

This article will show how to transmit video over RTP using the Toradex Linux Reference Multimedia Image and GStreamer. It focuses on using the H.264 encoding format.

Pre-requisites

Receiving H.264 Encoded RTP Video Stream on a Host Machine

In this section, learn how you can receive RTP video either using a GStreamer pipeline or VLC Player.

Using GStreamer

GStreamer is a low-latency method for receiving RTP video. On your host machine, install GStreamer and send the following command:

$ gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

Using Host PC: VLC Player

Optionally, you can use VLC player to receive RTP video on a PC. First, in your PC, create a sdp file with the following content:

stream.sdp
v=0
m=video 5000 RTP/AVP 96
c=IN IP4 127.0.0.1
a=rtpmap:96 H264/90000

After this, with the GStreamer pipepline on the device running, open this .sdp file with VLC Player on the host PC.


  • VLC Media Player

    VLC Media Player

Sending H.264 Encoded RTP Video Stream from SoM using GStreamer

In this section, learn how to send H.264-encoded video from a Toradex board over the network.

All Modules (SW encoding)

GStreamer provides an h.264 encoding element by software named x264enc. Use this plugin if your SoM does not support H.264 encoding by hardware or if you want to use the same pipeline on different modules. Note that the video performance will be lower compared with the plugins with encoding accelerated by hardware.

# gst-launch-1.0 videotestsrc ! videoconvert ! x264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000

Note: Replace <host-machine-ip> by the IP of the host machine.

i.MX8(X) Modules

Sending video pattern test frames:

# gst-launch-1.0 videotestsrc ! videoconvert ! v4l2h264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000

Note: Replace <host-machine-ip> by the IP of the host machine.

i.MX 8M Mini Quad/8M Plus Modules

Sending video pattern test frames:

# gst-launch-1.0 videotestsrc ! videoconvert ! vpuenc_h264 ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000

Note: Replace <host-machine-ip> by the IP of the host machine.

The Verdin iMX8M Mini DualLite doesn't have hardware encoding units, often named Video Processing Unit (VPU). Refer to the software encoding information.

i.MX6 Modules

Sending video pattern test frames:

# gst-launch-1.0 videotestsrc ! videoconvert ! vpuenc_h264 ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000

Note: Replace <host-machine-ip> by the IP of the host machine.

i.MX 6ULL/7 Modules

The i.MX 6ULL and i.MX 7 doesn't have hardware encoding units, often named Video Processing Unit (VPU). Refer to the software encoding information.

Note: In all examples above you can replace videotestsrc by v4l2src element to collect a stream from a camera.

Using Other Video Encoders

While examples of streaming video with other encoders are not provided, you may try it yourself. Use the gst-inspect tool to find available encoders and RTP payloaders on the board:

# gst-inspect-1.0 | grep -e "encoder"
# gst-inspect-1.0 | grep -e "rtp" -e " payloader"

Then browse the results and replace the elements in the original pipelines.

On the receiving end, you will have to use a corresponding payload. Inspect the payloader element to find the corresponding values. For example:

# gst-inspect-1.0 rtph264pay

You will find additional information on Video Encoding and Playback With GStreamer (Linux).