Search by Tags

Audio/Video over RTP (Linux)

 
Applicable for

Tags

Compare with Revision
Subscribe for this article updates

UDP/RTP Audio

Install prequisites (only necessary on BSP V1.x, the packages are already pre-installed on BSP V2.x)

root@colibri_t20:~# opkg install gst-plugin-audioconvert gst-plugin-audiotestsrc gst-plugin-rtp
Installing gst-plugin-audioconvert (0.10.31-r11.0.9) to root...
Downloading http://www.angstrom-distribution.org/feeds/next/ipk/eglibc/armv7a-vfp/gstreamer/gst-plugin-audioconvert_0.10.31-r11.0.9_armv7a-vfp.ipk.
Installing gst-plugin-audiotestsrc (0.10.31-r11.0.9) to root...
Downloading http://www.angstrom-distribution.org/feeds/next/ipk/eglibc/armv7a-vfp/gstreamer/gst-plugin-audiotestsrc_0.10.31-r11.0.9_armv7a-vfp.ipk.
Installing gst-plugin-rtp (0.10.26-r11.0.9) to root...
Downloading http://www.angstrom-distribution.org/feeds/next/ipk/eglibc/armv7a-vfp/gstreamer/gst-plugin-rtp_0.10.26-r11.0.9_armv7a-vfp.ipk.
Configuring gst-plugin-rtp.
Configuring gst-plugin-audiotestsrc.
Configuring gst-plugin-audioconvert.
root@colibri_t20:~# opkg --nodeps install gst-plugin-udp libgstnetbuffer-0.10-0
Installing gst-plugin-udp (0.10.26-r11.0.9) to root...
Downloading http://www.angstrom-distribution.org/feeds/next/ipk/eglibc/armv7a-vfp/gstreamer/gst-plugin-udp_0.10.26-r11.0.9_armv7a-vfp.ipk.
Installing libgstnetbuffer-0.10-0 (0.10.32-r11.1.9) to root...
Downloading http://www.angstrom-distribution.org/feeds/next/ipk/eglibc/armv5te/base/libgstnetbuffer-0.10-0_0.10.32-r11.1.9_armv5te.ipk.
Configuring libgstnetbuffer-0.10-0.
Configuring gst-plugin-udp.

Setting up receiver and sender.

root@colibri_t20:~# gst-launch -v udpsrc port=5000 ! "application/x-rtp,media=(string)audio, clock-rate=(int)44100, width=16, height=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96" ! rtpL16depay ! audioconvert ! alsasink device=hw:2,0 sync=false &
Setting pipeline to PAUSED ...
[ 4450.608338] i2s_set_channel_bit_count: enabling non-symmetric mode
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
root@colibri_t20:~# gst-launch audiotestsrc ! audioconvert ! audio/x-raw-int,channels=1,depth=16,width=16,rate=44100 ! rtpL16pay ! udpsink host=localhost port=5000
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, width=(int)16, height=(int)16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96
/GstPipeline:pipeline0/GstRtpL16Depay:rtpl16depay0.GstPad:src: caps = audio/x-raw-int, endianness=(int)4321, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100, channels=(int)1, channel-positions=(GstAudioChannelPosition)< GST_AUDIO_CHANNEL_POSITION_FRONT_MONO >
/GstPipeline:pipeline0/GstRtpL16Depay:rtpl16depay0.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, width=(int)16, height=(int)16, encoding-name=(string)L16, encoding-params=string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = audio/x-raw-int, endianness=(int)4321, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100, channels=(int)1, channel-positions=(GstAudioChannelPosition)< GST_AUDIO_CHANNEL_POSITION_FRONT_MONO >
/GstPipeline:pipeline0/GstAlsaSink:alsasink0.GstPad:sink: caps = audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100, channels=(int)2
^CCaught interrupt -- handling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 3755851999 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

UDP/RTP Audio & Video

First, one has to analyse the stream capabilities at the sender in order to be able to use them at the receiving end (press CTRL-C when status info has appeared):

root@colibri_t20:~# gst-launch -v filesrc location=/home/root/nv_medusa_h264_720_6M_cbr_2p_key60_q90_aac128_44.mp4 ! qtdemux name=demux demux.video_00 ! queue ! rtph264pay pt=96 ! udpsink host=localhost port=5000 demux.audio_00 ! queue ! rtpmp4apay pt=97 ! udpsink host=localhost port=5001
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-h264, level=(string)3.2, profile=(string)constrained-baseline, codec_data=(buffer)0142c020ffe100226742c020967402802dd80a0400002edc000afc80d18002dc8000b71d7bdf0784423501000468de3c80, width=(int)1280, height=(int)720, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-h264, level=(string)3.2, profile=(string)constrained-baseline, codec_data=(buffer)0142c020ffe100226742c020967402802dd80a0400002edc000afc80d18002dc8000b71d7bdf0784423501000468de3c80, width=(int)1280, height=(int)720, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0LAIJZ0AoAt2AoEAAAu3AAK/IDRgALcgAC3HXvfB4RCNQ\\=\\=\\,aN48gA\\=\\=\", payload=(int)96, ssrc=(uint)3857927412, clock-base=(uint)3864708562, seqnum-base=(uint)43861
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:sink: caps = video/x-h264, level=(string)3.2, profile=(string)constrained-baseline, codec_data=(buffer)0142c020ffe100226742c020967402802dd80a0400002edc000afc80d18002dc8000b71d7bdf0784423501000468de3c80, width=(int)1280, height=(int)720, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: timestamp = 3864708562
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: seqnum = 43861
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0LAIJZ0AoAt2AoEAAAu3AAK/IDRgALcgAC3HXvfB4RCNQ\\=\\=\\,aN48gA\\=\\=\", payload=(int)96, ssrc=(uint)3857927412, clock-base=(uint)3864708562, seqnum-base=(uint)43861
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)121056e500, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)121056e500, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstRtpMP4APay:rtpmp4apay0.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, encoding-name=(string)MP4A-LATM, cpresent=(string)0, config=(string)40002420adca00, payload=(int)97, ssrc=(uint)2324768575, clock-base=(uint)2616710116, seqnum-base=(uint)26397
/GstPipeline:pipeline0/GstRtpMP4APay:rtpmp4apay0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)121056e500, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstRtpMP4APay:rtpmp4apay0: timestamp = 2616710116
/GstPipeline:pipeline0/GstRtpMP4APay:rtpmp4apay0: seqnum = 26397
/GstPipeline:pipeline0/GstUDPSink:udpsink1.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, encoding-name=(string)MP4A-LATM, cpresent=(string)0, config=(string)40002420adca00, payload=(int)97, ssrc=(uint)2324768575, clock-base=(uint)2616710116, seqnum-base=(uint)26397
Pipeline is PREROLLED ...

Now we set environment variables from the udpsink0.GstPad:sink and udpsink1.GstPad:sink, probably ssrc, clock-base, seqnum-base can be omitted:

root@colibri_t20:~# VCAPS="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0LAIJZ0AoAt2AoEAAAu3AAK/IDRgALcgAC3HXvfB4RCNQ\\=\\=\\,aN48gA\\=\\=\", payload=(int)96"
root@colibri_t20:~# ACAPS="application/x-rtp, media=(string)audio, clock-rate=(int)44100, encoding-name=(string)MP4A-LATM, cpresent=(string)0, config=(string)40002420adca00, payload=(int)97"

Start receiver as a background process:

root@colibri_t20:~# gst-launch udpsrc port=5000 ! $VCAPS ! rtph264depay ! nv_omx_h264dec ! nv_gl_videosink rendertarget=0 udpsrc port=5001 ! $ACAPS ! rtpmp4adepay ! nv_omx_aacdec ! alsasink device=hw:2,0 &

Sender/Receiver on Localhost

Then start sender with the same command-line as above:

root@colibri_t20:~# gst-launch filesrc location=/home/root/nv_medusa_h264_720_6M_cbr_2p_key60_q90_aac128_44.mp4 ! qtdemux name=demux demux.video_00 ! queue ! rtph264pay pt=96 ! udpsink host=localhost port=5000 demux.audio_00 ! queue ! rtpmp4apay pt=97 ! udpsink host=localhost port=5001

Sending Point-to-Point

Of course it makes much more fun to steam over an actual real network. This time we run the sender on an Ubuntu Desktop 12.04 LTS 64-bit machine as follows:

gst-launch-0.10 filesrc location=nv_medusa_h264_720_6M_cbr_2p_key60_q90_aac128_44.mp4 ! qtdemux name=demux demux.video_00 ! queue ! rtph264pay pt=96 ! udpsink host=192.168.10.2 port=5000 demux.audio_00 ! queue ! rtpmp4apay pt=97 ! udpsink host=192.168.10.2 port=5001

Please note that the Colibri T20 target is assumed to have an assigned IP address of 192.168.10.2.

Sending Multicast

But the real eye catcher is when you stream the same video to multiple Colibri T20 targets!

First make sure to add a corresponding multicast route on the sender side as follows:

sudo route add -net 224.0.0.0 netmask 240.0.0.0 dev eth2

Where eth2 is the Ethernet interface towards your little Colibri T20 cluster.

And then stream just like before but now to a multicast address:

gst-launch-0.10 filesrc location=nv_medusa_h264_720_6M_cbr_2p_key60_q90_aac128_44.mp4 ! qtdemux name=demux demux.video_00 ! queue ! rtph264pay pt=96 ! udpsink host=224.0.0.1 port=5000    demux.audio_00 ! queue ! rtpmp4apay pt=97 ! udpsink host=224.0.0.1 port=5001

In order to get better results when streaming over a slightly congested network one can add an additional jitter buffer stage to the pipeline. But first things first, let's install such a Gstreamer plugin:

root@colibri_t20:~# opkg install gst-plugin-rtpmanager
Installing gst-plugin-rtpmanager (0.10.26-r11.0.9) to root...
Downloading http://feeds.angstrom-distribution.org/feeds/next/ipk/eglibc/armv7a-vfp/gstreamer/gst-plugin-rtpmanager_0.10.26-r11.0.9_armv7a-vfp.ipk.
Configuring gst-plugin-rtpmanager.

And then use it as follows:

root@colibri_t20:~# gst-launch udpsrc port=5000 ! $VCAPS ! gstrtpjitterbuffer ! rtph264depay ! nv_omx_h264dec ! nv_gl_videosink rendertarget=0 sync=false udpsrc port=5001 ! $ACAPS ! rtpmp4adepay ! nv_omx_aacdec ! alsasink device=hw:2,0