User:Mstone/Commentaries/Bundles 1
Note: Homunq's attempt to summarize the options after this discussion is at Bundles and updates.
Contents
Introduction
m_stone> (incidentally, my main goal for this conversation is to get us using
the same vocabulary.) To that end, I have a little picture I need to
draw. one second...
* - action
/ \
instance - / \
\ / \
*--/ /--*-------* - object container
/ //\\\
bundle / | \\\
/ | | \\
/ | | | \
* * * * * - objects
m_stone> This diagram represents the action of recording a roll of 10 photos
of butterflies in an instance of a recording activity derived from a
specified bundle. Key points: I can paint on a photo but not on a
record instance. On the other hand, I can use the instance to resume
recording butterflies or to locate an activity (included in a bundle,
perhaps with some other objects or even example instances) suitable
for beginning a new recording action on a different theme. (The
instance functions a bit like a UI continuation.)
Relations on Bytestreams
homunq> some wrangling about whether we want to do anything at all to group
"same activity" beyond signer. Eben says, no grouping. Same signer or
different activity. I think that will lead to too much pressure to
share keys - less security.
homunq> question: do we have a way to refer to two different bundles with
same signable content but different unsignable content (compression,
signatures, translatables)? (generally I'd like to ignore that case,
pretend "same bundle" == "same signature"). However, in the sugar
meeting I raised the issue of what version means (file format,
sharing protocol, config format) and got no reply.
m_stone> that's one of the important questions. From my perspective, we start
off with nothing but bytestreams. Our question is simply "what
information do we need to attach to bytestreams (and how?) in order
to get a desirable UI?". That's how I formulate the problem, anyway.
homunq> OK. Let me propose a term: "someone's activity" for the entire set of
versions that are signed by same key. Then we need a term for an
activity thread, all parents of a given version.
m_stone> You've suggested this notion of "someone's activity" before and I've
resisted it in the past because, while I think it's fine to group
several related bytestreams together, my inclination is not to regard
them as exchangeable at the lowest level.
homunq> agreed on the issue.
m_stone> On the whole, I like the "activity thread" metaphor, but here's an
important example. Suppose I've got some instances sitting in my DS.
These instances were clearly created by distinguished bytestreams.
When is it okay to garbage-collect (GC) those bytestreams?
homunq> Your question raises many sub-issues: formats, deleting old versions,
changing preferences, migrating instances,
homunq> so, terms are, activity thread, and unbroken activity thread ==
signed by same. our terms differ, but at least we agree on what is
important.
Private Data and Garbage Collection (GC)
homunq> One idea I had today: private journal entries; they should be keyed
to "unbroken activity thread" which created them (which is associated
with the bytestream that created them). If they were keyed to thread,
then sugar could decrypt them if it wanted but will only do so at the
behest of journal or that thread.
m_stone> are you imagining private journal entries, private instances, private
objects, or all three? Also, why do you want the access granularity
to be "unbroken thread" rather than "specific activity bytestream
<foobar>"?
homunq> urk that is a messy question. Easiest answer is all three or nothing.
As for why I prefer activity threads: because it is a royal pain to
have to explicitly switch bytestreams otherwise - unless you can
think of some UI?
m_stone> Why not drag the instance onto the bytestream you want?
homunq> For dozens of instances, between two different screens? yuk.
m_stone> When do you open dozens of instances at once on an XO?
homunq> not at once; sequentially, over weeks of time. still, yuk.
m_stone> also, if the human operator deleted the old bytestream, then sure,
use the new one. (This attitude is less convincing if it was
autodeleted.)
homunq> I think you do have to make things explicit when the activity thread
is broken.
m_stone> anyhow, we talked about this question for a long time at the
miniconference. many people felt as you do. I don't think they're
correct, but I don't really want to stand in the way.
homunq> lets talk about broken activity threads, because here is where I
think we will agree. I hope that if we can make something less-ugly
there, it will be acceptable everywhere.
m_stone> what identifies two bytestreams as belonging to the same thread?
homunq> The unvarnished claim of the later one, if the thread is broken. You
ready to discuss using this as a working assumption?
m_stone> well, I'd like to hear where you go with it. I'm not sold yet, but
it's a fine working hypothesis.
homunq> OK. So, the later one is claiming, "I can open instances". Instances
are private or not at creation. (each activity has a default and lets
users change default or change a given entry). A private instance is
"keyed to" (sugar refuses to open it for anything but) one or more
unbroken activity threads and always opens by default with most
recent version from thread that created it. If you try to delete that
version, sugar checks to see if anything "keyed to" that.
m_stone> how does this privacy protection differ in nature from the
P_DOCUMENT/P_DOCUMENT_RO protections?
homunq> P_DOCUMENT says "explicitly open". This is clarifying the definition
of "explicit" when multiple versions are present.
m_stone> why would we want to build a UI that was ever ambiguous about what
version to use?
homunq> not ambiguous, that is what we're defining. I can always "open with"
(though moving past a break in a thread shows something in red or
some warning indicator), but what if I explicitly delete an unbroken
portion of an activity thread for which I still have a later portion?
Magic or No Magic?
homunq> I think non-private entries should automatically open with whoever
claims to open them and private ones should refuse without an
explicit "open with". This makes "set someone else's later version as
default" == "manually delete old version" but for private entries
there is an additional step for security. one thing which I wonder -
do we want to have any kind of blacklist, ever?
m_stone> I just don't believe that we should resume instances with new
activity-bytestreams.
homunq> Even if we discover that bundle X is really an evil trojan which is
after your data? Or just broken? The implication of that is that we
almost never get to garbage collect old bundles.
homunq> With develop, you could easily be churning out dozens of debugging
versions a day? Think: each one would still have its instance in the
journal: "I ran and crashed" so you could never garbage collect any
of them.
m_stone> agreed, but I'm not sure how it's related to the current question -
you could GC them as soon as you deleted the failed instances that
were made from them. Why is this bad - it seems pretty good to me!
(Also, if you've got a school server, you could probably _archive_
the old versions more quickly.)
homunq> I guess I expect a little bit more magic.
m_stone> I agree the fundamental difference is that you expect "more magic"
and I expect "less magic"
homunq> It is really an open question for me, but disk space is an issue, I
think.
m_stone> it's definitely worth thinking about.
homunq> An activity with a bundled library could easily be 10s of k, 100
versions of that is starting to be appreciable.
m_stone> so you use up space rapidly, then you delete it when you're unhappy,
no? look, I can easily be wrong here. Maybe our particular audience
is better served with "more magic". However, I'm just not persuaded
yet, and I'm not the only person who needs to be persuaded one way or
another. Let's just note it as a point of contention and keep going.
homunq> Would you be happy if the degree of magic were configurable? The
default setting would obviously be "more", otherwise its not worth
the work. Would that make you happy?
m_stone> not particularly. (I don't mind being wrong about the appropriate
intensity of magic, but I see little immediate point in making it
configurable.)
homunq> If it is not easier to open an instance in another member of same
thread, thread has little point.
Where does signing fit in?
m_stone> Let's go back to activity threads. What exactly is being
signed/verified/asserted, by whom, when, and so on?
homunq> The executable bytestream (not including translatables) is signed.
Translatables are signed separately, by translators. Both are signed
with keys which are just data on somebody's XO (or elsewhere), not
with an actual XO key.
homunq> (I skimmed up on SPKI and agree that that is the only right way to do
more key management, but it would be hard to do it right.)
m_stone> I just wanted you to be aware that it existed. It's not clear that we
can afford to do it. :)
homunq> no gain unless you can tell (by asking a school server) what is an XO
key and what isn't. probably requires better connectivity in other
ways too. currently, I don't know if school servers even know the
answer to that question, btw, let alone can answer it.
m_stone> so what's the point of all this signing stuff?
homunq> what is the point? key is data, you get little help taking care of
it, but in an ideal world you manage to anyway. This is actually the
biggest question I wanted to ask you - what "little help" can we
give? Assuming you do take care of your keys, then the signatures
mean something. :( (not happy with that answer, but the only other
answer is SPKI.)
Auto Updates? No, just easy code sharing
m_stone> well, again, what are we hoping to achieve by signing things?
homunq> magic! auto updates. garbage collection. (GC is much easier if we
have auto-updates.)
m_stone> you've mentioned auto-updates before and I'm still don't believe that
we want them. (I'm sure that we want easy code sharing, but that
doesn't rely on auto-updates at all). In particular, who actually
wants auto-updates? Your best point is that you might be able to free
up some space by deleting some big activities. great. you'll also
break lots of instances because backwards compatibility is _really
hard_.
cjb> I think we want auto-updates so that I can join a shared activity
with a newer version than mine? ie., it's a special case of "I'm
joining an activity that I don't have installed."
m_stone> I've heard that reply and it's crap. You don't need to change what
activity bytestream is used in for new instances in order to share
code with someone. You just need a way for me to get your code onto
my XO and then for me to use it to join the specific shared instance
that I'm trying to join. So, again - who actually wants auto-updates?
cjb> I think that a user would confuse them Which is perhaps why I did. I
think that seeing BrandShinyNewMaze when I join a friend's and then
seeing OldCrappyMazeThatCorruptsData when I run it manually would be
very unintuitive.
homunq> you would still see brandshinynewmaze on the instance that you had
shared with your friend. we are also not talking about which activity
is favoritted.
m_stone> (incidentally, it just occurred to me that we _could_ start printing
version identifiers (or bytestring fragments) on or 'underneath'
activity icons)
cjb> I admit to having had the assumption that joining a shared instance
of a new activity was equal to upgrading that activity. However, I
think our users will find it very confusing that doesn't affect their
standard Maze activity. You're pushing package management onto our
users. (Who are not able to demonstrate, but I suspect would
demonstrate vigorously, that they don't want to have the
responsibility of choosing which versions of packages they want to
use.)
m_stone> cjb: well, that's the question, now isn't it - "more magic" or "less
magic"?
cjb> I think you're arguing for more magic than me. You're certainly
arguing for more code... that is, the "we will invent abstractions to
avoid the concept of an activity having newer versions of itself".
This seems more magical and less intuitive than catering to the
common case of someone having a newer activity for me to download and
install.
Bias toward new or old versions?
cjb> would the new activity become the favorited version for that activity
automatically? I guess that't the central question I'm interested in.
If you think the answer's no, then you're inflicting package
management on users.
m_stone> the central question is, indeed, are we biased toward newer versions?
Put differently, are newer versions 'stickier' than old ones?
homunq> favorited version = security threat for future actions? used for old
instances = security threat for old data?
m_stone> not exactly - it's symmetric from a security standpoint. sticking on
a bad old version is equally bad to automatically moving to bad new
versions. (or so it seems absent any real data)
homunq> if all versions are "sticky" there is no need for any signatures,
except as a UI element when signed by somebody you actually know. I
think newer should be stickier. I think that activities should be
explicit about which versions they say they can open, and we should
trust that. (not talking about security, just the ability to open). I
also think that the ability to open a given mime type will be pretty
stable, so the only risk is losing instances, which is not the end of
the world. I mean, we've all lived without them so far.
m_stone> I think that older should be stickier because I think it avoids
global breakage. (...physical discussion with cjb & Robot101
ensues...). cjb's basic point is that we should be extremely careful
about "complexifying" the UI. in particular, he's worried about
building UIs that demand understanding of pointers, which he feels is
currently true of the 'favorites' proposal. The present consensus
among cjb, robot101, and myself is that we should _not_ auto-update
activities but instead should give a UI hint when new versions are
detected. In other words, the UI cycle should consist of 'see
interesting new activity, explore interesting new activity, notice
hint to make new activity the default, then make it so'. Instances
should open with the activity that created them when possible. (we
didn't talk about what to do when you've deleted the activity that
old instances wanted to use.) one argument is that you shouldn't
delete an old activity until you've migrated its instances away.
another argument is that you simply become unable to resume the old
instance (though you can easily grab its object container). finally,
we could do magic, and pray that it works.
Robot101> Imagine a conversation: "your paint has more buttons" "yes I got a
new one, want to join?"
m_stone> We have good sidechannels for indicating that an activity is worth
using; namely, advertise it in word and deed. We have no such
advertising for identifying bad versions. Consequently, we should
prefer stability of version because this "quarantines breakage"
without impeding the spread of good activities through out-of-band
communication. (in other words, a good goal is to propagate working
things but not broken things)
Bundle Deletion Redux
homunq> I think that you need at least assisted cleanup. No automatic is OK.
cjb> I feel like all these arguments make our users hate us.
homunq> users? what about the developers? :)
cjb> "augh I can't resume my file because I have a slightly newer version
of an activity this sucks I hate you all"
m_stone> In my opinion, we shouldn't have garbage-collected your old activity
until you had no instances relying on it.
cjb> what about if I deleted it because it's big?
Robot101> I'd wonder about using the favourite version provided it's not older
than the version that created the instance, and the instance's
version otherwise(but this is possibly at odds with what the current
concept of favourite is; i.e., not every activity has a favourite
verson, just the ones which were selected to display on the ring)
m_stone> good suggestion. I like the idea of using the favorited version if we
break referential integrity by deleting the old activity while it
still has instances linked.
cjb> I also like that idea. I keep anticipating that someone who believes
that versioning is meaningless and we can't assume anything about
anything is going to come along and disagree with us. Pleasingly, it
doesn't seem to be happening.
Robot101> someone who has got a new version of an activity on their system is
unlikely to have arrived at that state without appreciating that new
versions of activities exist...
cjb> (..."Can you spell ref-er-ential int-egg-rity, Bobby?")
Robot101> it could arrive on their system by joining a newer shared instance
cjb> that's exactly the use case we're considering, AIUI
homunq> I like the basic schema. Missing: what help do we give for cleanup
(including UI hints)? What if activity thread is broken?
m_stone> ideally, your old activity would still be available from an archive.
as I said - I personally think it's okay if some instances become
non-resumable when you break them. (so long as it's easy to snag
their object container so that you can keep using the data you
generated in them)... (since I think that snagging object containers
will be a regular operation anyway)
homunq> m_stone++++++: - of course, offload!
Followup
homunq> OK, one little piece of UI sugar: when you set a new version as your
favorite, you say why. Others see that why when they are about to
share with you. Risks a lot of "newer newer really newer" but that is
not really bad.
m_stone> cute suggestion.
Robot101> user generated changelogs! :P
m_stone> I'd rather see more pervasive chat or wikis, but I don't have any
immediate objections...
cjb> OH MY GOD STOP. This is even worse than inflicting package
manaagement! now we're inflicting changelogs!
m_stone> Well, it is a "Journal". They often contain explanatory text
reflecting on the situations that caused one to take action...
m_stone> homunq: please send some mail to appropriate lists. leave holes where
you want me or cjb to try to things in. my goal is just to keep
building up written knowledge on the lists and wikis - we'll get
there soon enough. Eventually someone will decide that they
understand the problem well enough to take a crack at solving it. Or
they'll say "you idiots, look at this huge gaping hole in your
reasoning." Either way, progress will be made. :) (You certainly
pushed us hard enough to get some real feedback, which I think is
definitely a good thing).
m_stone> basically, keep up the good work!
