User:Mstone/Commentaries/Bundles 1

See also second Bundles commentary

Note: Homunq's attempt to summarize the options after this discussion is at Bundles and updates.

Introduction

 m_stone> (incidentally, my main goal for this conversation is to get us using
          the same vocabulary.) To that end, I have a little picture I need to
          draw. one second...

                         *         - action
                        / \
          instance -   /   \
                    \ /     \
             *--/ /--*-------*       - object container
            /              //\\\
        bundle            / | \\\
                         /  | | \\
                        /   | | | \
                       *    * * *  *    - objects

 m_stone> This diagram represents the action of recording a roll of 10 photos
          of butterflies in an instance of a recording activity derived from a
          specified bundle. Key points: I can paint on a photo but not on a
          record instance. On the other hand, I can use the instance to resume
          recording butterflies or to locate an activity (included in a bundle,
          perhaps with some other objects or even example instances) suitable
          for beginning a new recording action on a different theme. (The
          instance functions a bit like a UI continuation.)

Relations on Bytestreams

  homunq> some wrangling about whether we want to do anything at all to group
          "same activity" beyond signer. Eben says, no grouping. Same signer or
          different activity. I think that will lead to too much pressure to
          share keys - less security.

  homunq> question: do we have a way to refer to two different bundles with
          same signable content but different unsignable content (compression,
          signatures, translatables)? (generally I'd like to ignore that case,
          pretend "same bundle" == "same signature"). However, in the sugar
          meeting I raised the issue of what version means (file format,
          sharing protocol, config format) and got no reply.
 m_stone> that's one of the important questions. From my perspective, we start
          off with nothing but bytestreams. Our question is simply "what
          information do we need to attach to bytestreams (and how?) in order
          to get a desirable UI?". That's how I formulate the problem, anyway.

  homunq> OK. Let me propose a term: "someone's activity" for the entire set of
          versions that are signed by same key. Then we need a term for an
          activity thread, all parents of a given version.
 m_stone> You've suggested this notion of "someone's activity" before and I've
          resisted it in the past because, while I think it's fine to group
          several related bytestreams together, my inclination is not to regard
          them as exchangeable at the lowest level.
  homunq> agreed on the issue.

 m_stone> On the whole, I like the "activity thread" metaphor, but here's an
          important example. Suppose I've got some instances sitting in my DS.
          These instances were clearly created by distinguished bytestreams.
          When is it okay to garbage-collect (GC) those bytestreams?
  homunq> Your question raises many sub-issues: formats, deleting old versions,
          changing preferences, migrating instances,
  homunq> so, terms are, activity thread, and unbroken activity thread ==
          signed by same. our terms differ, but at least we agree on what is
          important.

Private Data and Garbage Collection (GC)

  homunq> One idea I had today: private journal entries; they should be keyed
          to "unbroken activity thread" which created them (which is associated
          with the bytestream that created them). If they were keyed to thread,
          then sugar could decrypt them if it wanted but will only do so at the
          behest of journal or that thread.
 m_stone> are you imagining private journal entries, private instances, private
          objects, or all three? Also, why do you want the access granularity
          to be "unbroken thread" rather than "specific activity bytestream
          <foobar>"?

  homunq> urk that is a messy question. Easiest answer is all three or nothing.
          As for why I prefer activity threads: because it is a royal pain to
          have to explicitly switch bytestreams otherwise - unless you can
          think of some UI?
 m_stone> Why not drag the instance onto the bytestream you want?
  homunq> For dozens of instances, between two different screens? yuk.
 m_stone> When do you open dozens of instances at once on an XO?
  homunq> not at once; sequentially, over weeks of time. still, yuk.

 m_stone> also, if the human operator deleted the old bytestream, then sure,
          use the new one. (This attitude is less convincing if it was
          autodeleted.)

  homunq> I think you do have to make things explicit when the activity thread
          is broken.
 m_stone> anyhow, we talked about this question for a long time at the
          miniconference. many people felt as you do. I don't think they're
          correct, but I don't really want to stand in the way.

  homunq> lets talk about broken activity threads, because here is where I
          think we will agree. I hope that if we can make something less-ugly
          there, it will be acceptable everywhere.

 m_stone> what identifies two bytestreams as belonging to the same thread?
  homunq> The unvarnished claim of the later one, if the thread is broken. You
          ready to discuss using this as a working assumption?
 m_stone> well, I'd like to hear where you go with it. I'm not sold yet, but
          it's a fine working hypothesis.
  homunq> OK. So, the later one is claiming, "I can open instances". Instances
          are private or not at creation. (each activity has a default and lets
          users change default or change a given entry). A private instance is
          "keyed to" (sugar refuses to open it for anything but) one or more
          unbroken activity threads and always opens by default with most
          recent version from thread that created it. If you try to delete that
          version, sugar checks to see if anything "keyed to" that.

 m_stone> how does this privacy protection differ in nature from the
          P_DOCUMENT/P_DOCUMENT_RO protections?
  homunq> P_DOCUMENT says "explicitly open". This is clarifying the definition
          of "explicit" when multiple versions are present.
 m_stone> why would we want to build a UI that was ever ambiguous about what
          version to use?
  homunq> not ambiguous, that is what we're defining. I can always "open with"
          (though moving past a break in a thread shows something in red or
          some warning indicator), but what if I explicitly delete an unbroken
          portion of an activity thread for which I still have a later portion?

Magic or No Magic?

  homunq> I think non-private entries should automatically open with whoever
          claims to open them and private ones should refuse without an
          explicit "open with". This makes "set someone else's later version as
          default" == "manually delete old version" but for private entries
          there is an additional step for security. one thing which I wonder -
          do we want to have any kind of blacklist, ever?
 m_stone> I just don't believe that we should resume instances with new
          activity-bytestreams.
  homunq> Even if we discover that bundle X is really an evil trojan which is
          after your data? Or just broken? The implication of that is that we
          almost never get to garbage collect old bundles.
  homunq> With develop, you could easily be churning out dozens of debugging
          versions a day? Think: each one would still have its instance in the
          journal: "I ran and crashed" so you could never garbage collect any
          of them.
 m_stone> agreed, but I'm not sure how it's related to the current question -
          you could GC them as soon as you deleted the failed instances that
          were made from them. Why is this bad - it seems pretty good to me!
          (Also, if you've got a school server, you could probably _archive_
          the old versions more quickly.)

  homunq> I guess I expect a little bit more magic.
 m_stone> I agree the fundamental difference is that you expect "more magic"
          and I expect "less magic"
  homunq> It is really an open question for me, but disk space is an issue, I
          think.
 m_stone> it's definitely worth thinking about.

  homunq> An activity with a bundled library could easily be 10s of k, 100
          versions of that is starting to be appreciable.
 m_stone> so you use up space rapidly, then you delete it when you're unhappy,
          no? look, I can easily be wrong here. Maybe our particular audience
          is better served with "more magic". However, I'm just not persuaded
          yet, and I'm not the only person who needs to be persuaded one way or
          another. Let's just note it as a point of contention and keep going.

  homunq> Would you be happy if the degree of magic were configurable? The
          default setting would obviously be "more", otherwise its not worth
          the work. Would that make you happy?
 m_stone> not particularly. (I don't mind being wrong about the appropriate
          intensity of magic, but I see little immediate point in making it
          configurable.)
  homunq> If it is not easier to open an instance in another member of same
          thread, thread has little point.

Where does signing fit in?

 m_stone> Let's go back to activity threads. What exactly is being
          signed/verified/asserted, by whom, when, and so on?
  homunq> The executable bytestream (not including translatables) is signed.
          Translatables are signed separately, by translators. Both are signed
          with keys which are just data on somebody's XO (or elsewhere), not
          with an actual XO key.
  homunq> (I skimmed up on SPKI and agree that that is the only right way to do
          more key management, but it would be hard to do it right.)
 m_stone> I just wanted you to be aware that it existed. It's not clear that we
          can afford to do it. :)
  homunq> no gain unless you can tell (by asking a school server) what is an XO
          key and what isn't. probably requires better connectivity in other
          ways too. currently, I don't know if school servers even know the
          answer to that question, btw, let alone can answer it.

 m_stone> so what's the point of all this signing stuff?
  homunq> what is the point? key is data, you get little help taking care of
          it, but in an ideal world you manage to anyway. This is actually the
          biggest question I wanted to ask you - what "little help" can we
          give? Assuming you do take care of your keys, then the signatures
          mean something. :( (not happy with that answer, but the only other
          answer is SPKI.)

Auto Updates? No, just easy code sharing

 m_stone> well, again, what are we hoping to achieve by signing things?
  homunq> magic! auto updates. garbage collection. (GC is much easier if we
          have auto-updates.)
 m_stone> you've mentioned auto-updates before and I'm still don't believe that
          we want them. (I'm sure that we want easy code sharing, but that
          doesn't rely on auto-updates at all). In particular, who actually
          wants auto-updates? Your best point is that you might be able to free
          up some space by deleting some big activities. great. you'll also
          break lots of instances because backwards compatibility is _really
          hard_.
     cjb> I think we want auto-updates so that I can join a shared activity
          with a newer version than mine? ie., it's a special case of "I'm
          joining an activity that I don't have installed."
 m_stone> I've heard that reply and it's crap. You don't need to change what
          activity bytestream is used in for new instances in order to share
          code with someone. You just need a way for me to get your code onto
          my XO and then for me to use it to join the specific shared instance
          that I'm trying to join. So, again - who actually wants auto-updates?

     cjb> I think that a user would confuse them Which is perhaps why I did. I
          think that seeing BrandShinyNewMaze when I join a friend's and then
          seeing OldCrappyMazeThatCorruptsData when I run it manually would be
          very unintuitive.
  homunq> you would still see brandshinynewmaze on the instance that you had
          shared with your friend. we are also not talking about which activity
          is favoritted.
 m_stone> (incidentally, it just occurred to me that we _could_ start printing
          version identifiers (or bytestring fragments) on or 'underneath'
          activity icons)
     cjb> I admit to having had the assumption that joining a shared instance
          of a new activity was equal to upgrading that activity. However, I
          think our users will find it very confusing that doesn't affect their
          standard Maze activity. You're pushing package management onto our
          users. (Who are not able to demonstrate, but I suspect would
          demonstrate vigorously, that they don't want to have the
          responsibility of choosing which versions of packages they want to
          use.)

 m_stone> cjb: well, that's the question, now isn't it - "more magic" or "less
          magic"?
     cjb> I think you're arguing for more magic than me. You're certainly
          arguing for more code... that is, the "we will invent abstractions to
          avoid the concept of an activity having newer versions of itself".
          This seems more magical and less intuitive than catering to the
          common case of someone having a newer activity for me to download and
          install.

Bias toward new or old versions?

     cjb> would the new activity become the favorited version for that activity
          automatically? I guess that't the central question I'm interested in.
          If you think the answer's no, then you're inflicting package
          management on users.
 m_stone> the central question is, indeed, are we biased toward newer versions?
          Put differently, are newer versions 'stickier' than old ones?

  homunq> favorited version = security threat for future actions? used for old
          instances = security threat for old data?
 m_stone> not exactly - it's symmetric from a security standpoint. sticking on
          a bad old version is equally bad to automatically moving to bad new
          versions. (or so it seems absent any real data)

  homunq> if all versions are "sticky" there is no need for any signatures,
          except as a UI element when signed by somebody you actually know. I
          think newer should be stickier. I think that activities should be
          explicit about which versions they say they can open, and we should
          trust that. (not talking about security, just the ability to open). I
          also think that the ability to open a given mime type will be pretty
          stable, so the only risk is losing instances, which is not the end of
          the world. I mean, we've all lived without them so far.

 m_stone> I think that older should be stickier because I think it avoids
          global breakage. (...physical discussion with cjb & Robot101
          ensues...). cjb's basic point is that we should be extremely careful
          about "complexifying" the UI. in particular, he's worried about
          building UIs that demand understanding of pointers, which he feels is
          currently true of the 'favorites' proposal. The present consensus
          among cjb, robot101, and myself is that we should _not_ auto-update
          activities but instead should give a UI hint when new versions are
          detected. In other words, the UI cycle should consist of 'see
          interesting new activity, explore interesting new activity, notice
          hint to make new activity the default, then make it so'. Instances
          should open with the activity that created them when possible. (we
          didn't talk about what to do when you've deleted the activity that
          old instances wanted to use.) one argument is that you shouldn't
          delete an old activity until you've migrated its instances away.
          another argument is that you simply become unable to resume the old
          instance (though you can easily grab its object container). finally,
          we could do magic, and pray that it works.

Robot101> Imagine a conversation: "your paint has more buttons" "yes I got a
          new one, want to join?"
 m_stone> We have good sidechannels for indicating that an activity is worth
          using; namely, advertise it in word and deed. We have no such
          advertising for identifying bad versions. Consequently, we should
          prefer stability of version because this "quarantines breakage"
          without impeding the spread of good activities through out-of-band
          communication. (in other words, a good goal is to propagate working
          things but not broken things)

Bundle Deletion Redux

  homunq> I think that you need at least assisted cleanup. No automatic is OK.
     cjb> I feel like all these arguments make our users hate us.
  homunq> users? what about the developers? :)
     cjb> "augh I can't resume my file because I have a slightly newer version
          of an activity this sucks I hate you all"
 m_stone> In my opinion, we shouldn't have garbage-collected your old activity
          until you had no instances relying on it.
     cjb> what about if I deleted it because it's big?
Robot101> I'd wonder about using the favourite version provided it's not older
          than the version that created the instance, and the instance's
          version otherwise(but this is possibly at odds with what the current
          concept of favourite is; i.e., not every activity has a favourite
          verson, just the ones which were selected to display on the ring)
 m_stone> good suggestion. I like the idea of using the favorited version if we
          break referential integrity by deleting the old activity while it
          still has instances linked.
     cjb> I also like that idea. I keep anticipating that someone who believes
          that versioning is meaningless and we can't assume anything about
          anything is going to come along and disagree with us. Pleasingly, it
          doesn't seem to be happening.
Robot101> someone who has got a new version of an activity on their system is
          unlikely to have arrived at that state without appreciating that new
          versions of activities exist...

     cjb> (..."Can you spell ref-er-ential int-egg-rity, Bobby?")
Robot101> it could arrive on their system by joining a newer shared instance
     cjb> that's exactly the use case we're considering, AIUI

  homunq> I like the basic schema. Missing: what help do we give for cleanup
          (including UI hints)? What if activity thread is broken?
 m_stone> ideally, your old activity would still be available from an archive.
          as I said - I personally think it's okay if some instances become
          non-resumable when you break them. (so long as it's easy to snag
          their object container so that you can keep using the data you
          generated in them)... (since I think that snagging object containers
          will be a regular operation anyway)
  homunq> m_stone++++++: - of course, offload!

Followup

  homunq> OK, one little piece of UI sugar: when you set a new version as your
          favorite, you say why. Others see that why when they are about to
          share with you. Risks a lot of "newer newer really newer" but that is
          not really bad.
 m_stone> cute suggestion.
Robot101> user generated changelogs! :P
 m_stone> I'd rather see more pervasive chat or wikis, but I don't have any
          immediate objections...
     cjb> OH MY GOD STOP. This is even worse than inflicting package
          manaagement!  now we're inflicting changelogs!
 m_stone> Well, it is a "Journal". They often contain explanatory text
          reflecting on the situations that caused one to take action...

 m_stone> homunq: please send some mail to appropriate lists. leave holes where
          you want me or cjb to try to things in. my goal is just to keep
          building up written knowledge on the lists and wikis - we'll get
          there soon enough. Eventually someone will decide that they
          understand the problem well enough to take a crack at solving it. Or
          they'll say "you idiots, look at this huge gaping hole in your
          reasoning." Either way, progress will be made. :) (You certainly
          pushed us hard enough to get some real feedback, which I think is
          definitely a good thing).
 m_stone> basically, keep up the good work!