Programming the camera: Difference between revisions

From OLPC
Jump to navigation Jump to search
(→‎Other resources: Creating Category:Camera.)
 
(24 intermediate revisions by 12 users not shown)
Line 1: Line 1:
{{Translations}}
This page explores how to interact with the laptop's built-in video camera.
This page explores how to interact with the laptop's built-in video camera.


Line 7: Line 8:
gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png
gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png


You'll need to run the above command in either a terminal in the developer console (<tt>alt-=</tt>) or one of the virtual terminals (e.g. <tt>ctrl-alt-f1</tt>). Note this means you don't need the Sugar GUI to be running to access the camera.
If you're typing that command in by hand, note that "v4l2src" begins with the characters vee-four-ell-two, an abbreviation for "Video 4 Linux 2.0", and not the number four hundred and twelve. You'll need to run the above command in a terminal, either the Terminal activity, in the developer console (<tt>alt-=</tt>) or one of the virtual terminals (e.g. <tt>ctrl-alt-f1</tt>). Note this means you don't need the Sugar GUI to be running to access the camera.


You can view the PNG image created as a result of the command in the Web activity.
You can view the PNG image created as a result of the command in the Web activity.
Line 13: Line 14:
Now, let's try and get some video on the screen:
Now, let's try and get some video on the screen:


gst-launch-0.10 v4l2src ! ximagesink
gst-launch-0.10 v4l2src ! xvimagesink


Unlike the first command this command will only work when executed in a terminal in the developer console. The resulting video will appear <i>behind </i>the developer console window so you'll need to move the developer console window aside to see the video.
Unlike the first command this command will only work with an X display, eg when executed in Terminal activity or a terminal in the developer console.


Since you've now had your first hit of "ooo, shiny" moving pictures let's take a look at what's happening behind the scenes.
Since you've now had your first hit of "ooo, shiny" moving pictures let's take a look at what's happening behind the scenes.
Line 21: Line 22:
== What just happened? ==
== What just happened? ==


I'm going to assume you have a passing familiarity with GStreamer, if not, you could go read about it. Basically, it's a series of pipes you can throw multimedia data down and get something in a file or on screen at the end. The data starts at a source (<tt>src</tt>) and ends up in a sink (<tt>sink</tt>) and can go through a number of intermediate manipulations along the way.
I'm going to assume you have a passing familiarity with [http://gstreamer.freedesktop.org/ GStreamer], if not, you could go read about it. Basically, it's a series of pipes you can throw multimedia data down and get something in a file or on screen at the end. The data starts at a source (<tt>src</tt>) and ends up in a sink (<tt>sink</tt>) and can go through a number of intermediate manipulations along the way.


While we will get to using the camera from Python eventually, we've started out with using the GStreamer command line tool <tt>gst-launch</tt>. The <tt>gst-launch</tt> tool is a quick way to experiment with putting a pipeline together and seeing what it does.
While we will get to using the camera from Python eventually, we've started out with using the GStreamer command line tool <tt>gst-launch</tt>. The <tt>gst-launch</tt> tool is a quick way to experiment with putting a pipeline together and seeing what it does.
Line 29: Line 30:
gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png
gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png


The camera in a XO laptop is a regular [http://linuxtv.org/v4lwiki/ Video4Linux 2] device which is accessed via GStreamer's <tt>v4l2src</tt> source--see that wasn't just a pile of random characters my cat threw up. Since the camera is our source it's the first item in our pipeline--notice the individual parts of the pipeline are separated with <tt>!</tt> characters. (You could say the data goes out with a bang but it'd be a pretty bad joke.)
The camera in a XO laptop is a regular [http://linuxtv.org/v4lwiki/ Video4Linux 2] device which is accessed via GStreamer's <tt>v4l2src</tt> source. Since the camera is our source it's the first item in our pipeline--notice the individual parts of the pipeline are separated with <tt>!</tt> characters. (You could say the data goes out with a bang but it'd be a pretty bad joke.)


Next, we'll skip to look at the end of the pipeline--the sink end--here we find the <tt>filesink</tt> which simply outputs some data to a particular file. The name of the file (in our case <tt>foo.png</tt>) is provided by specifying <tt>location=foo.png</tt>&mdash;this is an example of how to supply arguments to the individual items in the pipeline. (Note: if you try to use <tt>name=</tt> it won't work! The <tt>name</tt> parameter is for referring to the element in the pipeline, <i>not</i> the name of the destination file.)
Next, we'll skip to look at the end of the pipeline--the sink end--here we find the <tt>filesink</tt> which simply outputs some data to a particular file. The name of the file (in our case <tt>foo.png</tt>) is provided by specifying <tt>location=foo.png</tt>&mdash;this is an example of how to supply arguments to the individual items in the pipeline. (Note: if you try to use <tt>name=</tt> it won't work! The <tt>name</tt> parameter is for referring to the element in the pipeline, <i>not</i> the name of the destination file.)
Line 37: Line 38:
The only other item in the pipeline is the delightfully named <tt>ffmpegcolorspace</tt> plugin which performs colorspace conversions&mdash;essentially the <tt>v4l2src</tt> and <tt>pngenc</tt> plugins can't talk to each other directly because they each describe images in different ways, it's the job of <tt>ffmpegcolorspace</tt> to enable them to communicate by translating between the two styles of image description.
The only other item in the pipeline is the delightfully named <tt>ffmpegcolorspace</tt> plugin which performs colorspace conversions&mdash;essentially the <tt>v4l2src</tt> and <tt>pngenc</tt> plugins can't talk to each other directly because they each describe images in different ways, it's the job of <tt>ffmpegcolorspace</tt> to enable them to communicate by translating between the two styles of image description.


== Doing it in Python ==
<i>More to come...</i>


Implementing the same pipeline in Python might look like this:
== Other resources ==
* See sugar/shell/intro/glive.py for sample code [http://mediamods.com/public-svn/camera-activity/Camera.activity/glive.py]


import gst
import tempfile
GST_PIPE = ['v4l2src', 'ffmpegcolorspace', 'pngenc']
class Camera(object):
"""A class representing the OLPC camera."""
def __init__(self):
snap_file, self.snap_path = tempfile.mkstemp()
pipe = GST_PIPE + ['filesink location=%s' % self.snap_path]
self.pipe = gst.parse_launch('!'.join(pipe))
self.bus = self.pipe.get_bus()
def Snap(self):
"""Take a snapshot."""
self.pipe.set_state(gst.STATE_PLAYING)
self.bus.poll(gst.MESSAGE_EOS, -1)
if __name__ == '__main__':
c = Camera()
c.Snap()
print c.snap_path


==GStreamer 101==
----

Note: The author is learning this as he goes along, so items described may not be the best way or correct way to do anything. And he might get bored with writing this and stop halfw
Moved to [[GStreamer#GStreamer 101]].

==See also==
*[[:Category:Camera]]
*[[Vision processing]]
*http://gburt.blogspot.com/2007/11/one-laptop-per-photographer.html
*http://auricle.dyndns.org/ALE/ ALE is an image-processing program used for tasks such as image mosaicking, super-resolution, deblurring, noise reduction, anti-aliasing, and scene reconstruction. Its principle of operation is synthetic capture, combining multiple inputs representing the same scene.
*[[MotionDetection]]


[[Category:Developers]]
[[Category:Developers]]

Latest revision as of 07:41, 9 June 2015

  english | 한국어 HowTo [ID# 297406]  +/-  

This page explores how to interact with the laptop's built-in video camera.

Getting started

First, let's see the quickest way we can capture a still image from the camera--using a GStreamer command-line tool:

 gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png

If you're typing that command in by hand, note that "v4l2src" begins with the characters vee-four-ell-two, an abbreviation for "Video 4 Linux 2.0", and not the number four hundred and twelve. You'll need to run the above command in a terminal, either the Terminal activity, in the developer console (alt-=) or one of the virtual terminals (e.g. ctrl-alt-f1). Note this means you don't need the Sugar GUI to be running to access the camera.

You can view the PNG image created as a result of the command in the Web activity.

Now, let's try and get some video on the screen:

 gst-launch-0.10 v4l2src ! xvimagesink

Unlike the first command this command will only work with an X display, eg when executed in Terminal activity or a terminal in the developer console.

Since you've now had your first hit of "ooo, shiny" moving pictures let's take a look at what's happening behind the scenes.

What just happened?

I'm going to assume you have a passing familiarity with GStreamer, if not, you could go read about it. Basically, it's a series of pipes you can throw multimedia data down and get something in a file or on screen at the end. The data starts at a source (src) and ends up in a sink (sink) and can go through a number of intermediate manipulations along the way.

While we will get to using the camera from Python eventually, we've started out with using the GStreamer command line tool gst-launch. The gst-launch tool is a quick way to experiment with putting a pipeline together and seeing what it does.

Let's take a look at that first command line again:

 gst-launch-0.10 v4l2src ! ffmpegcolorspace ! pngenc ! filesink location=foo.png

The camera in a XO laptop is a regular Video4Linux 2 device which is accessed via GStreamer's v4l2src source. Since the camera is our source it's the first item in our pipeline--notice the individual parts of the pipeline are separated with ! characters. (You could say the data goes out with a bang but it'd be a pretty bad joke.)

Next, we'll skip to look at the end of the pipeline--the sink end--here we find the filesink which simply outputs some data to a particular file. The name of the file (in our case foo.png) is provided by specifying location=foo.png—this is an example of how to supply arguments to the individual items in the pipeline. (Note: if you try to use name= it won't work! The name parameter is for referring to the element in the pipeline, not the name of the destination file.)

As you can probably guess, the pngenc plugin is in the pipeline to convert the data from the video camera into the PNG file format before it is written to the the file.

The only other item in the pipeline is the delightfully named ffmpegcolorspace plugin which performs colorspace conversions—essentially the v4l2src and pngenc plugins can't talk to each other directly because they each describe images in different ways, it's the job of ffmpegcolorspace to enable them to communicate by translating between the two styles of image description.

Doing it in Python

Implementing the same pipeline in Python might look like this:

import gst
import tempfile

GST_PIPE = ['v4l2src', 'ffmpegcolorspace', 'pngenc']

class Camera(object):

  """A class representing the OLPC camera."""

  def __init__(self):
    snap_file, self.snap_path = tempfile.mkstemp()
    pipe = GST_PIPE + ['filesink location=%s' % self.snap_path]
    self.pipe = gst.parse_launch('!'.join(pipe))
    self.bus = self.pipe.get_bus()

  def Snap(self):
    """Take a snapshot."""
    self.pipe.set_state(gst.STATE_PLAYING)
    self.bus.poll(gst.MESSAGE_EOS, -1)

if __name__ == '__main__':
  c = Camera()
  c.Snap()
  print c.snap_path

GStreamer 101

Moved to GStreamer#GStreamer 101.

See also