Annotation Pointers

One of the more challenging parts of Annotation is handling the pointers into content.

Problem Space

Pointers into stable content are fairly simple. You could use a byte offset, an XPath expression, or possibly XPointer.

It becomes more difficult when the content changes. Where should the annotation point to then? How can we make the pointers resilient in that case?

Also, what kind of pointers do we need for different kinds of content?

Content-Specific Pointers

Different kinds of content will need different kinds of pointers. The kind of information in a pointer might be:

• An image: the coordinates you want to point to, a rectangular space you want to comment on, even an arbitrary shape that contains the commented portion of the image.
• A PDF: the page you are commenting on, and coordinates into the page (possibly represented as percentages). Or like an image you might have a shape. If you can parse the PDF, you could point to a range of text.
• HTML: a range of text, with start point and end point. A particular element, that may or may not have an id. A header.
• An audio file: a point in time, or range of time.
• A video file: a point in time, or range of time. Possibly with coordinates.

There's no single kind of pointer that applies to every kind of content. Some of the ways of pointing may be more sophisticated, and some clients may not be able to represent the pointers. For instance, a particular browser client may not be able to point into a time range of an embedded video.

Explicit Failure Handling

Ideally pointers will have documented ways they should handle failure cases. For instance, if you have a pixel coordinate that is larger than the given image, you should truncate that at the border of the image. A more resilient pointer would be percentages, which could be preserved even after an image resize.

Some pointer types might have to be adapted to make this work properly. For instance, XPath is really a query language. As such, the empty set is a valid result for any XPath expression. This doesn't really work for pointers, where as estimated result would be preferable. An example where this could happen is /html/body/div[2]/span[1] -- this selects the first span element in the second div element. If someone puts <strong> around that span element, then the expression will match nothing, even though the change in the content is relatively minor.

Pointers could contain preconditions to give a sense of their correctness. If the pointer isn't accurate, that could be indicated to the user so they could correct it. For instance, the original size of the image could be recorded, the aspect ratio, or other details that would detect the content has changed in an important way.

Redundant Pointers

When there are failures, redundancy through multiple kinds of pointers could help.

For instance, for the case of a comment on HTML you might include:

• An XPath expression
• The specific text (markup stripped) you were commenting on
• A percentage of the way down the page where you left your comment
• A nearby element that has an named anchor or id, plus some offset

This may imply that each pointer type should have a sense of *confidence*, so they can be compared.

(The text being commented on may be a separate part of the annotation, not attached to any particular pointer.)