Content stamping

The Problem

There's a lot of content out there, and we'd like a way to gauge the quality of an article. This is made harder by the multitude of often incompatible measures of quality.

Desirable characteristics of a solution:

• It should be non-intrusive -- i.e. not require work on the part of the content provider.
• It should be easy to set up a new group and provide article ratings.
• The process of setting up a group and submitting article ratings should be decoupled from that of using those ratings, making it both possible and safe for a new group to set up shop without running a gauntlet of credibility judges.

The Proposal

We describe a fairly general framework for collecting and disseminating/using information about content quality (article ratings). The anticipated use of the information is in generating views of the content, but we haven't explored that part in depth yet.

This scheme was originally aimed at Wikipedia, but it should be more broadly applicable. In particular, a simplified version of it might be useful in selecting content over the next few weeks for distribution on servers and laptops.

Concepts

• stamp -- A seal of approval that can be applied to an article. (This should probably be generalized into a numerical rating.)
• group -- A body of users authorized to apply a specific stamp.
• stamp depot -- A database of stamps and the groups responsible for them.
• valuation function -- A function that weighs an article's stamps and computes a scalar rating. (For something like Wikipedia, it might also take into account the article's categories, but that's a complication.)
• view -- The application of a valuation function to determine what to display. (For example, we might specify that an article is visible iff it has a positive valuation.)

Architecture

• Reviews are meta-data that can be stored in a repository completely independent of the content repository.

• Valuation functions can be created and maintained independently of both content and review repositories.

• Review depots can be created without any authorization, but a valuator will only consider stamps that it knows about, and it may demand some credentials or process. A review depot can set its own policy for review registration.

• Reviews could be considered specific to article revisions (i.e. automatically voided by unauthorized changes), but life is much simpler (and likely better) if we make them sticky (requiring explicit revocation in the wake of changes).

• Articles can be identified by URI.

• A stamp repository might have the following set of database tables:
  • reviews: (article-id, group-id, user-id, rating, comment, timestamp)
  • groups: (group-id, user-id)
  • users: (user-id, authentication info)

• It should be easy to add and remove (or deactivate) records and to retrieve the set of reviews associated with a given article. Naturally, mutative operations require some authentication.

Operational notes

• Groups must be administered, but how needn't be mandated. If a group is poorly administered, its stamp can be devalued.

• Stamp depots need to know about groups, and valuators need to know about stamp depots. Since each group is administered by a single stamp depot, any valuator that knows about that stamp depot can automatically discover new stamps.

Implementation

• Authentication might use OpenId.

• The database needn't be relational, but it might as well be.

Interface

• A user (somehow) selects a view, which (somehow) determines what he sees.

• If the view presents a non-current revision of an article, then an attempt to edit it is treated more or less like any attempt to edit a non-current revision.

• If you edit an article bearing a stamp that you are authorized to apply and which it already had, that stamp is automatically refreshed. (I.e. you're presumed to be doing no harm in your areas of authority.)

• It should be easy for members of a group to get notification of extra-group changes to an article bearing that group's stamp so they can either refresh or revoke the stamp.

• If you visit or edit an article bearing a stamp which you are authorized to apply but which is older than the current version, then the page should note this and encourage you to check the change and either refresh or revoke the stamp.

Starting Simply

• Use URIs for article-ids.
• Use a single stamp depot.
• Let stamps be sticky (i.e. not version-specific).
• Have a browser extension (or frame?) that, for each stamp you are authorized to apply, shows whether the current article has that stamp and lets you toggle it.
• Skip valuation functions for now; instead, just have another extension that shows what stamps the current article (page) has.
• The extensions might also indicate whether a stamp is stale (that is, older than the current version).