GStreamer

GStreamer is our multimedia framework. It is a library that allows the construction of graphs of media-handling components, ranging from simple Ogg/Vorbis playback to complex audio (mixing) and video (non-linear editing) processing.

http://gstreamer.freedesktop.org/

GStreamer 101

Examples

v4l2src can always be replaced by videotestsrc, and alsasrc by audiotestsrc.

Beginning of a normal video pipeline:

v4l2src ! queue ! videorate ! video/x-raw-yuv,framerate=15/1 ! videoscale ! video/x-raw-yuv,width=160,height=120 ! ...

Beginning of a normal audio pipeline:

alsasrc ! audio/x-raw-int,rate=8000,channels=1,depth=8 ! ...

Video encoding:

... ! ffmpegcolorspace ! theoraenc ! ...

Audio encoding:

... ! audioconvert ! vorbisenc ! ...

Video output:

... ! ffmpegcolorspace ! videoscale ! ximagesink

Audio output:

... ! audioconvert ! alsasink sync=false

Stdout:

... ! fdsink fd=1

Encode video+audio as ogg:

v4l2src ! queue ! ffmpegcolorspace ! theoraenc ! queue ! \
  oggmux name=mux  alsasrc ! queue ! audioconvert ! vorbisenc ! queue ! mux. mux. ! queue ! ...

A long version of videotestsrc ! ximagesink:

videotestsrc ! theoraenc ! oggmux ! oggdemux ! theoradec ! ffmpegcolorspace ! videoscale ! ximagesink

A long version of videotestsrc ! ximagesink &; audiotestsrc ! alsasink:

videotestsrc ! ffmpegcolorspace ! theoraenc ! queue ! \
  oggmux name=mux audiotestsrc ! audioconvert ! vorbisenc ! queue ! mux. mux. ! queue ! \
  oggdemux name=demux ! queue ! theoradec ! ffmpegcolorspace ! \
    ximagesink demux. ! queue ! vorbisdec ! audioconvert ! alsasink

Hangs on current fc6. MitchellNCharity 17:43, 12 June 2007 (EDT)

Live video streaming to an icecast server:

v4l2src ! ffmpegcolorspace ! videoscale ! video/x-raw-yuv,width=320,height=240 ! theoraenc quality=16 ! oggmux ! shout2send ip=192.168.1.100 port=8000 password=hackme mount=olpc.ogg

Live streaming to an icecast server:

v4l2src ! queue ! ffmpegcolorspace ! theoraenc quality=16 ! queue ! oggmux name=mux  alsasrc ! audio/x-raw-int,rate=8000,channels=1,depth=8 ! queue ! audioconvert ! vorbisenc ! queue ! mux. mux. ! \
  queue ! shout2send ip=... port=... password=... mount=/whatever.ogg

Notes

audio encoding

glive.py used wav rather than vorbis. Why?

Elements

Adapters:

• tee
• queue
• videoscale Convert video size.
• videorate Convert video rate.
• ffmpegcolorspace Convert video colorspace.
• audioconvert Convert audio format.

Video characteristics:

• video/x-raw-yuv
• framerate=15/1
• width=160,height=120

Audio characteristics:

• audio/x-raw-int,rate=8000,channels=1,depth=8

File I/O:

• fdsrc
• fdsink
• filesrc
• filesink
• gnomevfssrc
• gnomevfssink

Video sources:

• videotestsrc
• v4l2srcbroken

Audio sources:

• audiotestsrc
• alsasrc

Coding video:

• theoraenc
• theoradec

Coding audio:

• vorbisenc
• vorbisdec

Wrapping:

• oggmuxbroken (MitchellNCharity 11:44, 13 June 2007 (EDT))
• oggdemux

Outputing video:

• ximagesink or xvimagesink

Outputting audio:

• alsasink

Not sure about:

• vorbisparse

Sending ogg to an icecast streaming server:

... ! shout2send ip=... port=... password=... mount=/whatever.ogg
gst-launch ... ! fdsink | oggfwd host port password mount

Regretabbly, it appears fc6, and thus olpc, does not include shout2send in gst-plugins-good-plugins. Nor does it have rpms for oggfwd, or other possible alternatives. You can grab a random binary of oggfwd from it's site, or compile shout2send from source. :( MitchellNCharity 17:43, 12 June 2007 (EDT)

Correction, shout2send is on the olpc. oggfwd is not. --Damonkohler 23:14, 13 September 2007 (EDT)

Section TODO

• Fill in blank sections.
• Include snapshot recipes.
• Discuss python interfacing.

See also

• Programming the camera
• Video
• Sound
• http://gstreamer.freedesktop.org/
• http://gstreamer.freedesktop.org/documentation/
• Getting started with GStreamer with Python
• Software components ; Sugar Architecture/API#Third_Party_Packages

Other resources

• http://mediamods.com/public-svn/camera-activity/Camera.activity/glive.py
• https://coderanger.net/svn/projects/olpc/games/olpcgames/camera.py (said to not really be working (trouble knowing when gs is done) MitchellNCharity 18:24, 12 June 2007 (EDT))
• Where is the camera code in sugar? MitchellNCharity 18:16, 12 June 2007 (EDT)
• gstreamer-plugins
• gstreamer-plugins-plugin-coreelements
• gst-plugins-base-plugins
• gst-plugins-good-plugins
• google search "site:gstreamer.freedesktop.org"
• http://xiph.org/ Ogg/Theora/Vorbis