Content stamping

The Problem

There's a lot of content out there, and we'd like a way to gauge the quality of an article. This is made harder by the multitude of often incompatible measures of quality.

Desirable characteristics of a solution:

• It should be non-intrusive -- i.e. not require work on the part of the content provider.
• It should be easy to set up a new group of reviewers and review articles.
• The process of setting up a group and reviewing articles should be decoupled from that of using those reviews, making it both possible and safe for a new group to set up shop without running a gauntlet of credibility judges.

The Proposal

We describe a fairly general framework for collecting and disseminating/using information about content quality (reviews). The anticipated use of the information is in generating views of the content, but we haven't explored that part in depth yet.

This scheme was originally aimed at Wikipedia, but it should be more broadly applicable. In particular, a simplified version of it might be useful in selecting content over the next few weeks for distribution on servers and laptops.

Concepts

• group -- A registered body of reviewers.
• review -- A rating of an article by a user belonging to a specific group
• review depot -- A database of reviews and the groups responsible for them.
• valuation function -- A function that weighs an article's reviews and computes a numerical rating. (For something like Wikipedia, it might also take into account the article's categories, but that's a complication.)
• view -- The application of a valuation function to determine what to display. (For example, we might specify that an article is visible iff it has a positive valuation.)

Architecture

• Reviews are meta-data that can be stored in a repository completely independent of the content repository.

• Valuation functions can be created and maintained independently of both content and review repositories.

• Review depots can be created without any authorization, but a valuator will only consider groups that it knows about, and it may demand some credentials or process. A review depot can set its own policy for group registration.

• Reviews could be made specific to article revisions (i.e. automatically voided by unauthorized changes), but life is much simpler (and likely better) if we make them sticky (requiring explicit revocation in the wake of changes).

• Articles can be identified by URI.

• A stamp repository might have the following set of database tables:
  • reviews: (article-id, group-id, user-id, rating, comment, timestamp)
  • groups: (group-id, user-id)
  • users: (user-id, authentication info)

• It should be easy to add and remove (or deactivate) records and to retrieve the set of reviews associated with a given article. Naturally, mutative operations require some authentication.

Operational notes

• Groups must be administered, but how needn't be mandated. If a group is poorly administered, its reviews can be devalued.

• Stamp depots need to know about groups, and valuators need to know about stamp depots. Since each group is administered by a single stamp depot, any valuator that knows about that stamp depot can automatically discover new stamps.

Implementation

• Authentication might use OpenId.

• The database needn't be relational, but it might as well be.

Interface

• A user (somehow) selects a view, which (somehow) determines what he sees.

• If the view presents a non-current revision of an article, then an attempt to edit it is treated more or less like any attempt to edit a non-current revision.

• If you edit an article already reviewed by a group to which you belong, that review is automatically refreshed (i.e. its timestamp is updated). That is, you're presumed to be doing no harm in your areas of authority.

• It should be easy for members of a group to get notification of extra-group changes to an article reviewed by that group so that they can either refresh or alter that review.

• If you visit or edit an article with a 'stale review (i.e. which is older than the current version) that you are authorized to change, then the page should note this and encourage you to check the change and update the review.

Starting Simply

• Use URIs for article-ids.
• Use a single review depot.
• Let reviews be sticky (i.e. not version-specific).
• Have a browser extension (or frame?) that, for each group to which you belong, shows the current review of the article being viewed and lets you change it.
• Skip valuation functions for now; instead, just have another extension that shows what reviews the current article (page) has.
• The extensions might also indicate whether a review is stale.