Programming the camera
This page explores how to interact with the laptop's built-in video camera.
Getting started
First, let's see the quickest way we can capture a still image from the camera--using a GStreamer command-line tool:
gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png
You'll need to run the above command in either a terminal in the developer console (alt-=) or one of the virtual terminals (e.g. ctrl-alt-f1). Note this means you don't need the Sugar GUI to be running to access the camera.
You can view the PNG image created as a result of the command in the Web activity.
Now, let's try and get some video on the screen:
gst-launch-0.10 v4l2src ! ximagesink
Unlike the first command this command will only work when executed in a terminal in the developer console. The resulting video will appear behind the developer console window so you'll need to move the developer console window aside to see the video.
Since you've now had your first hit of "ooo, shiny" moving pictures let's take a look at what's happening behind the scenes.
What just happened?
I'm going to assume you have a passing familiarity with GStreamer, if not, you could go read about it. Basically, it's a series of pipes you can throw multimedia data down and get something in a file or on screen at the end. The data starts at a source (src) and ends up in a sink (sink) and can go through a number of intermediate manipulations along the way.
While we will get to using the camera from Python eventually, we've started out with using the GStreamer command line tool gst-launch. The gst-launch tool is a quick way to experiment with putting a pipeline together and seeing what it does.
Let's take a look at that first command line again:
gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png
The camera in a XO laptop is a regular Video4Linux 2 device which is accessed via GStreamer's v4l2src source--see that wasn't just a pile of random characters my cat threw up. Since the camera is our source it's the first item in our pipeline--notice the individual parts of the pipeline are separated with ! characters. (You could say the data goes out with a bang but it'd be a pretty bad joke.)
Next, we'll skip to look at the end of the pipeline--the sink end--here we find the filesink which simply outputs some data to a particular file. The name of the file (in our case foo.png) is provided by specifying location=foo.png—this is an example of how to supply arguments to the individual items in the pipeline. (Note: if you try to use name= it won't work! The name parameter is for referring to the element in the pipeline, not the name of the destination file.)
As you can probably guess, the pngenc plugin is in the pipeline to convert the data from the video camera into the PNG file format before it is written to the the file.
The only other item in the pipeline is the delightfully named ffmpegcolorspace plugin which performs colorspace conversions—essentially the v4l2src and pngenc plugins can't talk to each other directly because they each describe images in different ways, it's the job of ffmpegcolorspace to enable them to communicate by translating between the two styles of image description.
More to come...
Note: The author is learning this as he goes along, so items described may not be the best way or correct way to do anything. And he might get bored with writing this and stop halfw
GStreamer 101
Examples
v4l2src can always be replaced by videotestsrc, and alsasrc by audiotestsrc.
Beginning of a normal video pipeline:
v4l2src ! queue ! videorate ! video/x-raw-yuv,framerate=15/1 ! videoscale ! video/x-raw-yuv,width=160,height=120 ! ...
Beginning of a normal audio pipeline:
alsasrc ! audio/x-raw-int,rate=8000,channels=1,depth=8 ! ...
Video encoding:
... ! ffmpegcolorspace ! theoraenc ! ...
Audio encoding:
... ! audioconvert ! vorbisenc ! ...
Video output:
... ! ffmpegcolorspace ! videoscale ! ximagesink
Audio output:
... ! audioconvert ! alsasink sync=false
Stdout:
... ! fdsink fd=1
Encode video+audio as ogg:
v4l2src ! queue ! ffmpegcolorspace ! theoraenc ! queue ! \ oggmux name=mux alsasrc ! queue ! audioconvert ! vorbisenc ! queue ! mux. mux. ! queue ! ...
A long version of videotestsrc ! ximagesink:
videotestsrc ! theoraenc ! oggmux ! oggdemux ! theoradec ! ffmpegcolorspace ! videoscale ! ximagesink
A long version of videotestsrc ! ximagesink &; audiotestsrc ! alsasink:
videotestsrc ! ffmpegcolorspace ! theoraenc ! queue ! \
oggmux name=mux audiotestsrc ! audioconvert ! vorbisenc ! queue ! mux. mux. ! queue ! \
oggdemux name=demux ! queue ! theoradec ! ffmpegcolorspace ! \
ximagesink demux. ! queue ! vorbisdec ! audioconvert ! alsasink
- Hangs on current fc6. MitchellNCharity 17:43, 12 June 2007 (EDT)
Live streaming to an icecast server:
v4l2src ! queue ! ffmpegcolorspace ! theoraenc quality=16 ! queue ! oggmux name=mux alsasrc ! audio/x-raw-int,rate=8000,channels=1,depth=8 ! queue ! audioconvert ! vorbisenc ! queue ! mux. mux. ! \ queue ! shout2send ip=... port=... password=... mount=/whatever.ogg
Notes
video size, framerate, and theoraenc quality
audio encoding
glive.py used wav rather than vorbis. Why?
Elements
Adapters:
- tee
- queue
- videoscale Convert video size.
- videorate Convert video rate.
- ffmpegcolorspace Convert video colorspace.
- audioconvert Convert audio format.
Video characteristics:
Audio characteristics:
File I/O:
- fdsink (link currently broken MitchellNCharity 17:43, 12 June 2007 (EDT))
- filesrc
- filesink
- gnomevfssrc
- gnomevfssink
Video sources:
Audio sources:
Coding video:
Coding audio:
Wrapping:
- oggmux
- oggdemux
Outputing video:
Outputting audio:
Not sure about:
Sending ogg to an icecast streaming server:
... ! shout2send ip=... port=... password=... mount=/whatever.ogg gst-launch ... ! fdsink | oggfwd host port password mount
- Regretabbly, it appears fc6, and thus olpc, does not include shout2send in gst-plugins-good-plugins. Nor does it have rpms for oggfwd, or other possible alternatives. You can grab a random binary of oggfwd from it's site, or compile shout2send from source. :( MitchellNCharity 17:43, 12 June 2007 (EDT)
Doing live streaming video from an xo
Section TODO
- Fill in blank sections.
- Include snapshot recipes.
- Discuss python interfacing.
Other resources
- http://mediamods.com/public-svn/camera-activity/Camera.activity/glive.py
- https://coderanger.net/svn/projects/olpc/games/olpcgames/camera.py
- Where is the camera code in sugar? MitchellNCharity 18:16, 12 June 2007 (EDT)
- gstreamer-plugins
- gstreamer-plugins-plugin-coreelements
- gst-plugins-base-plugins
- gst-plugins-good-plugins
- google search "site:gstreamer.freedesktop.org"
- http://xiph.org/ Ogg/Theora/Vorbis