Note: Homunq's attempt to summarize the options after this discussion is at Bundles and updates.
m_stone> (incidentally, my main goal for this conversation is to get us using the same vocabulary.) To that end, I have a little picture I need to draw. one second...
* - action / \ instance - / \ \ / \ *--/ /--*-------* - object container / //\\\ bundle / | \\\ / | | \\ / | | | \ * * * * * - objects
m_stone> This diagram represents the action of recording a roll of 10 photos of butterflies in an instance of a recording activity derived from a specified bundle. Key points: I can paint on a photo but not on a record instance. On the other hand, I can use the instance to resume recording butterflies or to locate an activity (included in a bundle, perhaps with some other objects or even example instances) suitable for beginning a new recording action on a different theme. (The instance functions a bit like a UI continuation.)
Relations on Bytestreams
homunq> some wrangling about whether we want to do anything at all to group "same activity" beyond signer. Eben says, no grouping. Same signer or different activity. I think that will lead to too much pressure to share keys - less security.
homunq> question: do we have a way to refer to two different bundles with same signable content but different unsignable content (compression, signatures, translatables)? (generally I'd like to ignore that case, pretend "same bundle" == "same signature"). However, in the sugar meeting I raised the issue of what version means (file format, sharing protocol, config format) and got no reply. m_stone> that's one of the important questions. From my perspective, we start off with nothing but bytestreams. Our question is simply "what information do we need to attach to bytestreams (and how?) in order to get a desirable UI?". That's how I formulate the problem, anyway.
homunq> OK. Let me propose a term: "someone's activity" for the entire set of versions that are signed by same key. Then we need a term for an activity thread, all parents of a given version. m_stone> You've suggested this notion of "someone's activity" before and I've resisted it in the past because, while I think it's fine to group several related bytestreams together, my inclination is not to regard them as exchangeable at the lowest level. homunq> agreed on the issue.
m_stone> On the whole, I like the "activity thread" metaphor, but here's an important example. Suppose I've got some instances sitting in my DS. These instances were clearly created by distinguished bytestreams. When is it okay to garbage-collect (GC) those bytestreams? homunq> Your question raises many sub-issues: formats, deleting old versions, changing preferences, migrating instances, homunq> so, terms are, activity thread, and unbroken activity thread == signed by same. our terms differ, but at least we agree on what is important.
Private Data and Garbage Collection (GC)
homunq> One idea I had today: private journal entries; they should be keyed to "unbroken activity thread" which created them (which is associated with the bytestream that created them). If they were keyed to thread, then sugar could decrypt them if it wanted but will only do so at the behest of journal or that thread. m_stone> are you imagining private journal entries, private instances, private objects, or all three? Also, why do you want the access granularity to be "unbroken thread" rather than "specific activity bytestream <foobar>"?
homunq> urk that is a messy question. Easiest answer is all three or nothing. As for why I prefer activity threads: because it is a royal pain to have to explicitly switch bytestreams otherwise - unless you can think of some UI? m_stone> Why not drag the instance onto the bytestream you want? homunq> For dozens of instances, between two different screens? yuk. m_stone> When do you open dozens of instances at once on an XO? homunq> not at once; sequentially, over weeks of time. still, yuk.
m_stone> also, if the human operator deleted the old bytestream, then sure, use the new one. (This attitude is less convincing if it was autodeleted.)
homunq> I think you do have to make things explicit when the activity thread is broken. m_stone> anyhow, we talked about this question for a long time at the miniconference. many people felt as you do. I don't think they're correct, but I don't really want to stand in the way.
homunq> lets talk about broken activity threads, because here is where I think we will agree. I hope that if we can make something less-ugly there, it will be acceptable everywhere.
m_stone> what identifies two bytestreams as belonging to the same thread? homunq> The unvarnished claim of the later one, if the thread is broken. You ready to discuss using this as a working assumption? m_stone> well, I'd like to hear where you go with it. I'm not sold yet, but it's a fine working hypothesis. homunq> OK. So, the later one is claiming, "I can open instances". Instances are private or not at creation. (each activity has a default and lets users change default or change a given entry). A private instance is "keyed to" (sugar refuses to open it for anything but) one or more unbroken activity threads and always opens by default with most recent version from thread that created it. If you try to delete that version, sugar checks to see if anything "keyed to" that.
m_stone> how does this privacy protection differ in nature from the P_DOCUMENT/P_DOCUMENT_RO protections? homunq> P_DOCUMENT says "explicitly open". This is clarifying the definition of "explicit" when multiple versions are present. m_stone> why would we want to build a UI that was ever ambiguous about what version to use? homunq> not ambiguous, that is what we're defining. I can always "open with" (though moving past a break in a thread shows something in red or some warning indicator), but what if I explicitly delete an unbroken portion of an activity thread for which I still have a later portion?
Magic or No Magic?
homunq> I think non-private entries should automatically open with whoever claims to open them and private ones should refuse without an explicit "open with". This makes "set someone else's later version as default" == "manually delete old version" but for private entries there is an additional step for security. one thing which I wonder - do we want to have any kind of blacklist, ever? m_stone> I just don't believe that we should resume instances with new activity-bytestreams. homunq> Even if we discover that bundle X is really an evil trojan which is after your data? Or just broken? The implication of that is that we almost never get to garbage collect old bundles. homunq> With develop, you could easily be churning out dozens of debugging versions a day? Think: each one would still have its instance in the journal: "I ran and crashed" so you could never garbage collect any of them. m_stone> agreed, but I'm not sure how it's related to the current question - you could GC them as soon as you deleted the failed instances that were made from them. Why is this bad - it seems pretty good to me! (Also, if you've got a school server, you could probably _archive_ the old versions more quickly.)
homunq> I guess I expect a little bit more magic. m_stone> I agree the fundamental difference is that you expect "more magic" and I expect "less magic" homunq> It is really an open question for me, but disk space is an issue, I think. m_stone> it's definitely worth thinking about.
homunq> An activity with a bundled library could easily be 10s of k, 100 versions of that is starting to be appreciable. m_stone> so you use up space rapidly, then you delete it when you're unhappy, no? look, I can easily be wrong here. Maybe our particular audience is better served with "more magic". However, I'm just not persuaded yet, and I'm not the only person who needs to be persuaded one way or another. Let's just note it as a point of contention and keep going.
homunq> Would you be happy if the degree of magic were configurable? The default setting would obviously be "more", otherwise its not worth the work. Would that make you happy? m_stone> not particularly. (I don't mind being wrong about the appropriate intensity of magic, but I see little immediate point in making it configurable.) homunq> If it is not easier to open an instance in another member of same thread, thread has little point.
Where does signing fit in?
m_stone> Let's go back to activity threads. What exactly is being signed/verified/asserted, by whom, when, and so on? homunq> The executable bytestream (not including translatables) is signed. Translatables are signed separately, by translators. Both are signed with keys which are just data on somebody's XO (or elsewhere), not with an actual XO key. homunq> (I skimmed up on SPKI and agree that that is the only right way to do more key management, but it would be hard to do it right.) m_stone> I just wanted you to be aware that it existed. It's not clear that we can afford to do it. :) homunq> no gain unless you can tell (by asking a school server) what is an XO key and what isn't. probably requires better connectivity in other ways too. currently, I don't know if school servers even know the answer to that question, btw, let alone can answer it.
m_stone> so what's the point of all this signing stuff? homunq> what is the point? key is data, you get little help taking care of it, but in an ideal world you manage to anyway. This is actually the biggest question I wanted to ask you - what "little help" can we give? Assuming you do take care of your keys, then the signatures mean something. :( (not happy with that answer, but the only other answer is SPKI.)
Auto Updates? No, just easy code sharing
m_stone> well, again, what are we hoping to achieve by signing things? homunq> magic! auto updates. garbage collection. (GC is much easier if we have auto-updates.) m_stone> you've mentioned auto-updates before and I'm still don't believe that we want them. (I'm sure that we want easy code sharing, but that doesn't rely on auto-updates at all). In particular, who actually wants auto-updates? Your best point is that you might be able to free up some space by deleting some big activities. great. you'll also break lots of instances because backwards compatibility is _really hard_. cjb> I think we want auto-updates so that I can join a shared activity with a newer version than mine? ie., it's a special case of "I'm joining an activity that I don't have installed." m_stone> I've heard that reply and it's crap. You don't need to change what activity bytestream is used in for new instances in order to share code with someone. You just need a way for me to get your code onto my XO and then for me to use it to join the specific shared instance that I'm trying to join. So, again - who actually wants auto-updates?
cjb> I think that a user would confuse them Which is perhaps why I did. I think that seeing BrandShinyNewMaze when I join a friend's and then seeing OldCrappyMazeThatCorruptsData when I run it manually would be very unintuitive. homunq> you would still see brandshinynewmaze on the instance that you had shared with your friend. we are also not talking about which activity is favoritted. m_stone> (incidentally, it just occurred to me that we _could_ start printing version identifiers (or bytestring fragments) on or 'underneath' activity icons) cjb> I admit to having had the assumption that joining a shared instance of a new activity was equal to upgrading that activity. However, I think our users will find it very confusing that doesn't affect their standard Maze activity. You're pushing package management onto our users. (Who are not able to demonstrate, but I suspect would demonstrate vigorously, that they don't want to have the responsibility of choosing which versions of packages they want to use.)
m_stone> cjb: well, that's the question, now isn't it - "more magic" or "less magic"? cjb> I think you're arguing for more magic than me. You're certainly arguing for more code... that is, the "we will invent abstractions to avoid the concept of an activity having newer versions of itself". This seems more magical and less intuitive than catering to the common case of someone having a newer activity for me to download and install.
Bias toward new or old versions?
cjb> would the new activity become the favorited version for that activity automatically? I guess that't the central question I'm interested in. If you think the answer's no, then you're inflicting package management on users. m_stone> the central question is, indeed, are we biased toward newer versions? Put differently, are newer versions 'stickier' than old ones?
homunq> favorited version = security threat for future actions? used for old instances = security threat for old data? m_stone> not exactly - it's symmetric from a security standpoint. sticking on a bad old version is equally bad to automatically moving to bad new versions. (or so it seems absent any real data)
homunq> if all versions are "sticky" there is no need for any signatures, except as a UI element when signed by somebody you actually know. I think newer should be stickier. I think that activities should be explicit about which versions they say they can open, and we should trust that. (not talking about security, just the ability to open). I also think that the ability to open a given mime type will be pretty stable, so the only risk is losing instances, which is not the end of the world. I mean, we've all lived without them so far.
m_stone> I think that older should be stickier because I think it avoids global breakage. (...physical discussion with cjb & Robot101 ensues...). cjb's basic point is that we should be extremely careful about "complexifying" the UI. in particular, he's worried about building UIs that demand understanding of pointers, which he feels is currently true of the 'favorites' proposal. The present consensus among cjb, robot101, and myself is that we should _not_ auto-update activities but instead should give a UI hint when new versions are detected. In other words, the UI cycle should consist of 'see interesting new activity, explore interesting new activity, notice hint to make new activity the default, then make it so'. Instances should open with the activity that created them when possible. (we didn't talk about what to do when you've deleted the activity that old instances wanted to use.) one argument is that you shouldn't delete an old activity until you've migrated its instances away. another argument is that you simply become unable to resume the old instance (though you can easily grab its object container). finally, we could do magic, and pray that it works.
Robot101> Imagine a conversation: "your paint has more buttons" "yes I got a new one, want to join?" m_stone> We have good sidechannels for indicating that an activity is worth using; namely, advertise it in word and deed. We have no such advertising for identifying bad versions. Consequently, we should prefer stability of version because this "quarantines breakage" without impeding the spread of good activities through out-of-band communication. (in other words, a good goal is to propagate working things but not broken things)
Bundle Deletion Redux
homunq> I think that you need at least assisted cleanup. No automatic is OK. cjb> I feel like all these arguments make our users hate us. homunq> users? what about the developers? :) cjb> "augh I can't resume my file because I have a slightly newer version of an activity this sucks I hate you all" m_stone> In my opinion, we shouldn't have garbage-collected your old activity until you had no instances relying on it. cjb> what about if I deleted it because it's big? Robot101> I'd wonder about using the favourite version provided it's not older than the version that created the instance, and the instance's version otherwise(but this is possibly at odds with what the current concept of favourite is; i.e., not every activity has a favourite verson, just the ones which were selected to display on the ring) m_stone> good suggestion. I like the idea of using the favorited version if we break referential integrity by deleting the old activity while it still has instances linked. cjb> I also like that idea. I keep anticipating that someone who believes that versioning is meaningless and we can't assume anything about anything is going to come along and disagree with us. Pleasingly, it doesn't seem to be happening. Robot101> someone who has got a new version of an activity on their system is unlikely to have arrived at that state without appreciating that new versions of activities exist...
cjb> (..."Can you spell ref-er-ential int-egg-rity, Bobby?") Robot101> it could arrive on their system by joining a newer shared instance cjb> that's exactly the use case we're considering, AIUI
homunq> I like the basic schema. Missing: what help do we give for cleanup (including UI hints)? What if activity thread is broken? m_stone> ideally, your old activity would still be available from an archive. as I said - I personally think it's okay if some instances become non-resumable when you break them. (so long as it's easy to snag their object container so that you can keep using the data you generated in them)... (since I think that snagging object containers will be a regular operation anyway) homunq> m_stone++++++: - of course, offload!
homunq> OK, one little piece of UI sugar: when you set a new version as your favorite, you say why. Others see that why when they are about to share with you. Risks a lot of "newer newer really newer" but that is not really bad. m_stone> cute suggestion. Robot101> user generated changelogs! :P m_stone> I'd rather see more pervasive chat or wikis, but I don't have any immediate objections... cjb> OH MY GOD STOP. This is even worse than inflicting package manaagement! now we're inflicting changelogs! m_stone> Well, it is a "Journal". They often contain explanatory text reflecting on the situations that caused one to take action...
m_stone> homunq: please send some mail to appropriate lists. leave holes where you want me or cjb to try to things in. my goal is just to keep building up written knowledge on the lists and wikis - we'll get there soon enough. Eventually someone will decide that they understand the problem well enough to take a crack at solving it. Or they'll say "you idiots, look at this huge gaping hole in your reasoning." Either way, progress will be made. :) (You certainly pushed us hard enough to get some real feedback, which I think is definitely a good thing). m_stone> basically, keep up the good work!