Content stamping

The idea of 'stamping' content with a review by a group to which one belongs -- and having membership in a number of groups to allow one to apply such a stamp -- gave rise to the system below. We are moving towards using the term "review" instead of "stamp" for clarity.

The broad concept is that at any given time, a piece of content or work would have a number of reviews associated with it and with a review group. Within a review group, we assume there is communication, so that the latest review from any member of that group is taken to represent that group's opinion. A variety of views, giving different weight to different groups' reviews, can then be created for a library.

The Problem

There's a lot of content out there, and we'd like a way to gauge the quality of an article.

There are many dimensions for measuring content and quality, often overlapping. For instance, simplicity, conciseness, grammar, informativeness, entertainment value, age appropriateness, visual beauty, and bibliographic depth, can all be evaluated separately for a work... moreover, different people would be interested in and qualified to assess a work in each of these dimensions.

There are conflicting yet equally valid prioritizations of different dimensions of quality.

Desirable characteristics of a solution:

• Non-intrusive -- reviewing should require no work on the part of the content provider.
• Low barrier to entry -- it should be easy for a new group of reviewers to start reviewing works and revising their internal system, without proving something about themselves
• Diversity of standards -- groups that disagree strongly on principles should be able to review the same works without conflict.
• Diversity of views -- anyone should be able to create a custom view based on the available stamps and reviews.
• Separation of reviews and views -- The process of setting up a group and reviewing articles should be decoupled from that of using those reviews. This way the creation of new groups does not disrupt existing views.

The Proposal

We describe a fairly general framework for collecting and disseminating/using information about content quality (reviews). The anticipated use of the information is in generating views of the content, but we haven't explored that part in depth yet.

This scheme was originally aimed at Wikipedia, but it should be more broadly applicable. In particular, a simplified version of it might be useful in selecting content over the next few weeks for distribution on servers and laptops.

Concepts

• group -- A registered body of reviewers.
• review -- A rating of a work by a user belonging to a specific group
• review depot -- A database of reviews and the groups responsible for them.
• valuation function -- A function that weighs a work's reviews and computes a numerical rating. (For something like Wikipedia, it might also take into account the article's categories, but that's a complication.)
• view -- The application of a valuation function to determine what to display. (For example, we might specify that an article is visible iff it has a positive valuation.)
• stale review -- a review that applies to an older version of a work

Architecture

• Reviews are meta-data that can be stored in a repository completely independent of the content repository.

• Groups must register; this process should be fairly automatic and not arduous, but in place to prevent gaming particular views.

• Valuation functions can be created and maintained independently of both content and review repositories.

• Review depots can be created without any authorization, but a valuator will only consider groups that it knows about, and it may demand some credentials or process. A review depot can set its own policy for group registration.

• Reviews could be made specific to article revisions (i.e. automatically voided by unauthorized changes), but life is much simpler (and likely better) if we make them sticky (requiring explicit revocation in the wake of changes).

• Articles can be identified by URI.

• A review repository might have the following set of database tables:
  • reviews: (article-id, group-id, user-id, rating, comment, timestamp)
  • groups: (group-id, user-id)
  • users: (user-id, authentication info)

• It should be easy to add and remove (or deactivate) records and to retrieve the set of reviews associated with a given article. Naturally, mutative operations require some authentication.

Operational notes

Groups must be administered, but the mechanism for this need not be mandated. If a group is poorly administered, its reviews can be devalued.

Review depots need to know about groups, and valuators need to know about review depots. Since each group is administered by a single review depot, any valuator that knows about that review depot can automatically discover new reviews.

Implementation

• Authentication might use OpenId.

• The database needn't be relational, but it might as well be.

Interface

• A user (somehow) selects a view, which (somehow) determines what he sees.

• If the view presents a non-current revision of an article, then an attempt to edit it is treated more or less like any attempt to edit a non-current revision.

• If you edit an article already reviewed by a group to which you belong, that review is automatically refreshed (i.e. its timestamp is updated). That is, you're presumed to be doing no harm in your areas of authority.

• It should be easy for members of a group to get notification of extra-group changes to an article reviewed by that group so that they can either refresh or alter that review.

• If you visit or edit an article with a stale review (i.e. which is older than the current version) that you are authorized to change, then the page should note this and encourage you to check the change and update the review.

Starting Simply

• Use URIs for article-ids.
• Use a single review depot.
• Let reviews be sticky (i.e. not version-specific).
• Have a browser extension (or frame?) that, for each group to which you belong, shows the current review of the article being viewed and lets you change it.
• Skip valuation functions for now; instead, just have another extension that shows what reviews the current article (page) has.
• The extensions might also indicate whether a review is stale.