Journal and Overlays

This is the beginning of working up how to integrate Union File Systems and the Journal. Using AUFS-type overlays it should be possible to support both Sugar-specific and legacy applications in a fairly natural way as far as integrating into the Journal mechanism.

Journal Population (automated)

on start (pre-launch) create an overlay tree like so:

• core-system image
• application's installation image
• opened file r/o image (empty initially)
• current r/w image (empty initially)

on individual file open:

• if present on the system already
  • hard link the individual file from origin into r/o image
• otherwise acquire and store in the r/o image directory
• file name choice
  • auto-resolve (original + suffix if necessary)?
  • or on conflict choose overwrite/rename (allows you to "revert" to a previous version of the file)

on open journal (project name-space):

• load all files from journal record in r/o layer of image
  • allow for conflict resolution
• then allow selection of the particular file to open

on file save:

• metadata entry optional (tags for journal)
• stores as a name within the journal's "project space"
• no special instrumentation required for legacy applications

on application close (handled by overlay/chroot manager):

• create Journal entry (with application information)
  • record referenced file uris (files loaded into space)
  • record (real, underlying) filenames in journal for new/changed files
  • do not move or alter files on disk
• does cut/copy and paste produce a "reference"?
• allow Journal notes/metadata and the like

Backup of Journal

on backup

• record backup uri/uuid for the file + local file
• record encryption key for file (encrypted w/ the uri)
• upload encrypted package with metadata and changed files
• record backup (tentative) flag (should have a signed return-receipt from the top-level backup server before we trust that it's really "safe")

Sharing Journal Entries

The Journal should allow for sharing individual files or Journal entries (project name-spaces) with individuals, groups (e.g. classes or groups of friends or project-mates) or the whole world.

Sharing Operation

• if local, share file data + uri + document key
• if not local, share uri + document key
• default no share
  • "class" share option (need to find your class)
• sign request to share for backup
• encrypt sharing certificate for target users if not public
• what about the underlying files?
  • do we want one uuid and key per file? yes
  • select which files to share rather than the whole environment
• journal entry is itself a sharable file
  • share referenced resources? (when we own them) question on share
  • what about derivative works? Does the original source need to sign off on sharing?
• publish sharing certificate via whatever means
  • email
  • direct share via IM
  • publish on server
    • personal log channels (tags)
  • references fields
  • pass to someone else to forward

Comments

• references field in certificate
• normally text payload, but allow any content type
• share resource as normal

Blog/Syndication Interfaces

• display public postings
• allow filter by tag and date
• comments inserted by web "user" into separate account with references fields
• private comments shared directly with user

Server Considerations

Backup Issues

• needs to know who, how much,etc
• should only give to:
  • backup server (signed cert)
  • if desperate, someone who sees a backup server regularly
  • requires a mechanism to track connections

school server issues

• how to signal laptop that we are ready to go?
  • don't want 300 uploads at 08:00 and none at 08:01
  • especially as users will be starting working
  • do backup when we have local bandwidth spare (prefer, but get it done)
• how to prioritise uploads/downloads?
  • how urgent is your upload?
  • how much have you transferred already?
• how to serve files?
  • https server?
  • uploads as https file uploads?
• quotas need to be dynamic
  • on cache full no more accepted
  • who maintains on failure?
• "I have 25MB for backup, what do you want to store?"

Download Issues

• signed sharing cert
• uuid of already shared (fail if not avail)
• public-log-view
• uuid needs to include account information

Global Access Mechanism

• where does the root server go to get the data for a request?
  • same place it went to put it in, it must have the password to the gmail account to be recording the data in the first place
  • the fact that one server is storing those passwords is a possible issue
• filter out those for whom it doesn't sharing certificates
• every machine should have their data backup account password
  • central server for "outside" access
  • (plus) every machine have access software for self-account

Teacher's View of Logs

• that shared with the teacher (just as with any other person)
• if not public it's private, so the child doesn't automatically give access to the teacher
• comments to child are text notes in the teacher's journal shared with child
• marks and the like may or may not be shared
• default tag-set for each course (to allow children to subscribe to a given rss feed for each course)
• default "friends" group for each course

Scarce-resource Optimisations

no-backup/backup-until/backup-until-read-by

• voicemail, IM
  • what about abuse?
• server-level
  • allow for prioritisation in resource scarcity situations (upload bandwidth or space)
  • backup when possible, but drop first when full
• default in certain types to quick deletion or until-read
• take file size into account for strategy

Alternate Storage

• some simple way to suggest
  • diff-based storage
  • replacing storage
  • versioned storage
• plus file size hints
• plus timeout hints

Personal Importance Level

• prioritise in scarce resource situations
• automatically give precedence to last version of same resource within my uri space
  • (saved to same file as reference)
• references increase priority if not explicitly set

Journal Based File Open

• choose whole journal entry (namespace)
  • then choose file from it
• browse into entry to choose single file to import into new space
• browse journal to find old versions
• filter journal to find items (tag, type, text, etc)