Vmeta

vMeta is a hardware-accelerated video encoding and decoding engine included in the XO-1.75 and XO-4.

Installation and test

Tested on 12.1.0 on XO-1.75.

1. Install gstreamer-plugins-marvell and libvmeta-marvell
2. Download http://techslides.com/demos/sample-videos/small.mp4 and play it in Totem in GNOME

For a beefier test video, try http://archive.org/download/hellmans_mayonnaise_commercial/hellmans_mayonnaise_commercial.mpeg

Note that these packages do include software decoders too (not sure why) so just seeing the video play back doesn't mean that it is being done by the vmeta hardware engine. The best way to be confident that hardware accel is being used is by looking at totem standard output, which should be limited to:

vmetadec istance(0x....) found
At finalizing, Gstvmetadec instance(0x....) DecErroOccured is 0x0
Gstvmetadec instance(0x....) is finalized.

Also look in dmesg, check that there are not any vmeta-related complaints there.

For a more direct gstreamer test:

 gst-launch filesrc location=hellmans_mayonnaise_commercial.mpeg ! decodebin ! vmetaxvimagesink

Troubleshooting

Hangs after displaying first frame

Check that regular audio playback is working (e.g. with speaker-test).

Software component breakdown

uio_vmeta

This is an in-kernel UIO driver that allows the vmeta hardware engine to be driven from userspace (registers and interrupts). Some changes to the UIO core are needed as well.

libphycontmem

This library provides a userspace API to allocate and deal with a chunk of physically contiguous DMA-capable main memory. The buffer needs to be quite large, at least 32mb, 40mb in order to view higher quality videos.

Marvell code drops include backends for pmem and bmm (Buffer Management Module, probably a special kernel driver that provides such a buffer). An ion backend is also available from OLPC. This remains a sticky area.

• pmem was really designed to deal with fixed memory areas (e.g. graphics memory), but vmeta is something that we see as optional and on-demand; we don't want to hold this allocation even while vmeta is not being used, and we don't even know that vmeta will be used.
  • When OLPC shipped this for 13.1.0 and previous, we unconditionally set up a DMA-coherent allocation in early boot.
  • For 13.2.0 and newer, OLPC ships this solution for XO-1.75, but hacked to allocate the buffer only upon first use of pmem.
• ion is a little better, when combined with the DMA backend (which naturally uses CMA), now the allocation is done on-demand. However, ion provides no API to pass physical addresses to userspace, which is a requirement here.
  • OLPC ships this for XO-4, plus a hack (a new ioctl) to provide physical addresses to userspace.
• OLPC has not explored the BMM approach (to the best of my knowledge); this does seem to have been abandoned in newer Marvell code drops though.
• An ideal approach would be for UIO to expose on-demand contiguous DMA allocations; while that has been discussed/proposed a few times, no implementation has gone upstream.

libvmeta

This library provides an API for interacting with the low-level vmeta hardware: management of DMA buffers, power, clocks, etc. It uses libphycontmem for memory management and interacts with the uio_vmeta kernel interface for the rest.

marvell-ipp

   Continuing the tradition that IPP is lumped into the vmeta category...:

The IPP libraries are a set of proprietary software encoders/decoders. That makes them unrelated to vmeta, which is a hardware-based engine. There is also an overlap, the IPP codecs support some video formats that vmeta can play without software help. Shipping software codecs alongside a hardware-based decoding/encoding engine sounds odd but there is some basis for it:

• The vmeta specifications place some restrictions on video parameters for the supported formats. Maybe there are some files that vmeta cannot play, even though the base format is (generally) supported.
• vMeta only accelerates video, the audio stream must be decoded in software.
• These decoders are optimized for the hardware; we have versions that use iwMMXt2 and other versions that use NEON.
• While there is some overlap, the IPP codecs do support some video formats that are not supported by the hardware (e.g. wmv).

libvmetahal and libcodecvmeta

Further confusing the IPP software vs hardware situation, marvell-ipp (software codec package) ships the libraries required for hardware-accelerated vmeta playback. libcodecvmeta{dec,enc} link against libvmetahal. These are closed source libraries provided by Marvell.

gst-plugins-marvell

This is a set of gstreamer plugins that provide playback via vmeta or with the IPP codecs. libgstvmetadec links against libcodecvmetadec, and the IPP-based elements link against the IPP libraries.

gstreamer-plugins-marvell

This RPM package is built from two source tarballs: gst-plugins-marvell and gst-plugins-vmetaxv.

gst-plugins-marvell

This codebase seems to originate from Marvell. It provides several gstreamer plugins that do encoding/decoding using libvmeta (and friends) - apparently making existing gstreamer apps use hardware acceleration (via vmeta) for encoding and decoding purposes.

One source of this code is the Cubox package bundle (see their wiki) which includes gst-plugins-marvell0.10_0.2.0-0ubuntu1~ppa14.tar.gz. This source tarball has full git history from the developers at Marvell, with the latest change committed in March 2011.

An alternative codebase can be found in the private Android code-drops from Marvell. No git history is included, but I have reconstructed the history over different code drops at /home/dsd/private_git/gst-plugins-marvell-android. Looking at the files and changes through each code drop, it would appear that this codebase shares an origin with the codebase found in the cubox download, but later diverged, while still merging in some of the changes from the cubox-acquired git tree.

For example, the Android codebase includes a debian/changelog file, with the latest entry from 2009. The cubox codebase comes with the same changelog but with entries going all the way to 2011. However, some of the things described in the post-2009 cubox codebase changelog are implemented in the Android codebase. This suggests a fork of the codebase and some shared/confused history.

Despite the confusion, it does seem clear that this Android codebase is more recent than the Cubox tarball (with last commit date March 2011). The gingerbread-beta3 code drop was received in October 2011, and this same codebase was updated in the toughpad-a1 code drop received in November 2012. This new version has some small fixes and other changes.

The toughpad-a1 android codebase is also included (with no changes) in the qseven alpha4 code drop.

gstreamer-plugins-marvell-0.10-1

No source was published by the developer. Details unknown.

gstreamer-plugins-marvell-0.10-2

Our RPM was based on Cubox's gst-plugins-marvell0.10_0.2.0-0ubuntu1~ppa14.tar.gz, with this code change applied by OLPC. There is no associated info/changelog for this patch; presumably it is the minimum needed to make it work on MMP2. Here is the equivalant changed based on top of the git history from Marvell found inside the tarball.

gstreamer-plugins-marvell-0.10-3

There were no changes to the gst-plugins-marvell source used to build this version.

gstreamer-plugins-marvell-0.10-4

This version switches to the codebase found in the qseven alpha4 drop. Although there is no real history, I set up a git repo for our work at /home/dsd/private_git/gst-plugins-marvell-qseven.

OLPC's changes on top of the code drop are:

• g711dec and g711enc compilation enabled in configure.ac
• Marvell code compiles mpeg2dec with the FORBID_WMMX_DEINTERLACE flag; we drop that flag.
• Add missing include to fix compile of gstwmvdec
• Enable compilation of libgstvmetaenc (looks like the Marvell codebase only compiles the decoders)
• Update to the latest GMutex API to fix compile warnings/failures

vmetaxv

This source package is a fork of gst-plugins-base. It seems to originate from Marvell, under the name "gst-plugins-bmmxv" (Buffer Management Module), and is shipped in the Cubox package bundle. The source tarball gst-plugins-bmmxv0.10_0.10.25-0ubuntu1~ppa9.tar.gz included there includes git history. Here is a diff of the changes between gst-plugins-base-0.10.26.3 and the bmmxv tarball.

It looks like the motivation here is that xvimagesink always does a copy of the data it is going to display. bmmxvimagesink is a modified xvimagesink that, in the case where it detects that the buffer comes from vmeta, it avoids the memcpy. The physical address is sent to the Xv driver and it is handled directly.

OLPC has renamed/repackaged this as vmetaxvimagesink based on top of a newer gst-plugins-base commit. The history has been constructed from source tarball drops at gst-plugins-vmetaxv.

gstreamer-plugins-marvell-0.10-2.olpc

This is the first version where vmetaxvimagesink appears. It is based on gst-plugins-base git commit 12ef907f8a3762685da0b96391edc30a78d31805.

gstreamer-plugins-marvell-0.10-3.olpc

This is a minor change upon the previous release, which appears to be more careful when distinguishing between vmeta and non-vmeta image buffers.

gstreamer-plugins-marvell-0.10-4.xo4

Catches any error returned from posix_memalign (previously any failure here was ignored).

libvmeta-marvell

This package contains what is referred to by Marvell as libvmeta and marvell-ipp.

libvmeta

libvmeta is a simple library that provides access to vmeta kernel-level functionality. It provides:

1. Memory mapping functions, to allocate a region of DMA-capable memory (e.g. via pmem)
2. Access to the kernel's UIO interface to vmeta

libvmeta source code can be found in the Cubox package bundle. libvmeta_1.0ubuntu2.tar.gz has full git history up to December 2010 and uses bmm-lib for memory allocation. bmm-lib sources can also be found in the Cubox package bundle; it seems to use an unknown /dev/bmm interface to obtain DMA memory allocations, and also has some (not built/used) code that can do the same via Android's pmem.

libvmeta source code can also be found in the private Marvell Android code drops. The changes in each code drop have been imported into /home/dsd/private_git/libvmeta-android. The initial version used bmm-lib, but this was later updated to use pmem directly (in gingerbread-beta1). In gingerbread-beta3, this code was updated to use libphycontmem. This code drop also includes libphycontmem which appears to be a simple wrapper around pmem.

libvmeta source is also found in the qseven alpha4 code drop, the code is identical to the toughpad-a1 android version.

OLPC's libvmeta-marvell-005 offering links against libphycontmem, with pmem in place on the kernel side. The origins of this code are unclear.

libvmeta-marvell-006 is based on the qseven-alpha4 drop. libvmeta links against libphycontmem and libionhelper. A git tree to track OLPC's changes (currently trivial) is at /home/dsd/private_git/libvmeta-qseven.

marvell-ipp

This package seems to include a number of codecs. libcodecvmetadec is what calls into vmeta so that hardware-based decoding happens. The other codecs are software codecs which make use of IWMMXT2 for extra performance. It is not clear to me why software codecs are included in a hardware accelerated multimedia encoding/decoding system. One possibility (should be easy to test) is that these libraries are uploaded to the vmeta DSP and executed there.

libvmeta-marvell-005

In libvmeta-marvell-005 we ship the following armv5tel softfp libraries originating from "marvell-ipp":

• libcodec{aac,amrnb,amrwb,h263,h264,jpeg,mpeg4,vmeta}{dec,enc}
• libcodec{g711,gifdec,midi,mp3dec,mpeg2dec,pngdec,wmadev,wmvdec}
• libipp{cam,deint,ie,sp,vp}
• libvmetahal

And some hardfp libraries:

• libmiscgen

The exact origins of these are unknown. The Cubox bundle does include these libraries, but they are different, and all are softfp. The libmiscgen source code can be found inside marvell-ipp_0.2.1-0ubuntu1~ppa10.tar.gz at marvell-ipp/example/misc (this may or may not correspond to what OLPC actually ships here).

libvmeta-marvell-006

This release adds libcodecg729 (hardfp), and the following previously-softfp libraries are replaced by ARMv7/hardfp versions:

• libcodec{aacenc,aacdec,amrnb,amrwb,h264dec,mp3dec,mpeg4dec,wmadec,g711}

All of the other libraries have been changed as well (no matching md5sums) even though they are still softfp.

The source of all these libraries is the Marvell qseven alpha4 code drop.