Build system meeting minutes
Jump to navigation
Jump to search
Oct 25 16:00:40 <gregdek> OK, it's 4pm by my clock. Who's around to talk build system?
Oct 25 16:00:50 * notting (i=notting@redhat/notting) has joined #olpc-meeting
Oct 25 16:01:11 >marcopg< marco?
Oct 25 16:01:32 <gregdek> From the Fedora side, I see notting, mbonnet, spevack, dgilmore...
Oct 25 16:01:43 >marcopg< there've been two standing meetings here earlier
Oct 25 16:01:57 * spevack waves
Oct 25 16:02:06 * dgilmore is here
Oct 25 16:02:10 <gregdek> ...and from the OLPC side, _bernie, who's here for you guys?
Oct 25 16:02:29 >marcopg< jg's idea seems to be "let's write a script that builds stuff from srpms"
Oct 25 16:02:53 <_bernie> gregdek: just a moment...
Oct 25 16:03:01 <gregdek> OK.
Oct 25 16:03:04 * gregdek waits.
Oct 25 16:03:06 <_bernie> gregdek: jg and cscott are still in another meetling
Oct 25 16:03:33 * gregdek is still peeling green makeup from his skin.
Oct 25 16:03:57 * f13 (i=jkeating@fedora/ender) has joined #olpc-meeting
Oct 25 16:04:08 <dgilmore> gday f13
Oct 25 16:04:13 <f13> howdy
Oct 25 16:05:49 <gregdek> f13: still waiting for more folks.
Oct 25 16:05:58 <f13> ok.
Oct 25 16:09:08 <gregdek> _bernie: Any luck?
Oct 25 16:09:53 * jg (n=jg@wireless-142.media.mit.edu) has joined #olpc-meeting
Oct 25 16:10:01 <jg> sorry we're late...
Oct 25 16:10:03 <gregdek> Hey jg.
Oct 25 16:10:06 <_bernie> cscott is coming too
Oct 25 16:10:07 <gregdek> No problem. :)
Oct 25 16:10:17 <gregdek> Shall we wait for cscott?
Oct 25 16:10:20 <jg> life's a "bit" crazy.
Oct 25 16:10:25 <gregdek> I would imagine. :)
Oct 25 16:10:33 <jg> production starts one week from tomorrow, set the context.
Oct 25 16:10:35 <marcopg> gregdek: sounds like a good idea yeah
Oct 25 16:10:38 <jg> The train is moving down the tracks.
Oct 25 16:10:54 * carrano (n=chatzill@1cc-dhcp-81.media.mit.edu) has joined #olpc-meeting
Oct 25 16:10:56 <gregdek> Which is why I'm glad you can make the time.
Oct 25 16:11:12 <m_stone> good afternoon!
Oct 25 16:11:22 <m_stone> my apologies for the delay.
Oct 25 16:11:32 <gregdek> :)
Oct 25 16:11:59 <gregdek> _bernie: ready to start at your go-ahead.
Oct 25 16:12:41 <m_stone> we have one person we need to pry-bar out of his current discussion.
Oct 25 16:13:07 <gregdek> We could provide cake.
Oct 25 16:13:13 <gregdek> No, I guess not, actually.
Oct 25 16:14:26 <_bernie> :-)
Oct 25 16:14:34 <_bernie> cscott seems to be still stuck
Oct 25 16:15:02 * Mitch_Bradley (i=wmb@wireless-119.media.mit.edu) has joined #olpc-meeting
Oct 25 16:15:25 <gregdek> 4:15... is it safe to get started without him?
Oct 25 16:15:37 * m_stone starts looking for his fire-ax
Oct 25 16:15:41 <_bernie> he's saying 2 mins
Oct 25 16:15:43 <m_stone> gregdek: unfortunately, no.
Oct 25 16:15:48 <_bernie> sorry very much for the delay
Oct 25 16:16:15 <gregdek> No problem.
Oct 25 16:16:15 * _sj| has quit (Read error: 104 (Connection reset by peer))
Oct 25 16:16:42 <_bernie> well, before we start
Oct 25 16:16:57 <_bernie> how about using the original list of issues as a starting point?
Oct 25 16:17:24 <_bernie> earlier today we had a "standing up meeting" where a few more issues came up
Oct 25 16:17:28 <gregdek> Yeah, I'm happy with that. Do you want to restate them?
Oct 25 16:17:34 * jg gets a fireax....
Oct 25 16:17:46 <_bernie> gregdek: how about I list them one at a time and we discuss them?
Oct 25 16:17:52 <gregdek> Sure.
Oct 25 16:18:12 * c_scott (n=cscott@wireless-19-74.media.mit.edu) has joined #olpc-meeting
Oct 25 16:18:24 <_bernie> hellO!
Oct 25 16:18:25 <m_stone> our prodigal son returns.
Oct 25 16:18:28 <jg> the gang' all here...
Oct 25 16:18:35 <f13> oh good
Oct 25 16:18:36 <_bernie> jg: or maybe it would be better if you moderated?
Oct 25 16:18:42 <m_stone> shall we do introductions first?
Oct 25 16:18:45 <c_scott> hello, i'm dizzy but here
Oct 25 16:19:01 <f13> introductions work for me.
Oct 25 16:19:03 <jg> OK, I'm Jim Gettys, V.P. Software, OLPC....
Oct 25 16:19:16 <gregdek> Greg DeKoenigsberg, community development, Red Hat.
Oct 25 16:19:19 <jg> infamous for X11, http, handhelds.org
Oct 25 16:19:23 <spevack> Max Spevack, Fedora Project
Oct 25 16:19:47 <m_stone> I'm Michael Stone. I do security stuff and work on all the infrastructure (build-system, testing, ...) required to make that go.
Oct 25 16:20:08 <m_stone> (for OLPC)
Oct 25 16:20:08 <c_scott> I'm C. Scott Ananian, I fix things that are broken.
Oct 25 16:20:10 <c_scott> (for OLPC)
Oct 25 16:20:14 <m_stone> (by which me means that he breaks things)
Oct 25 16:20:18 <jg> that too.
Oct 25 16:20:20 <gregdek> Tough room. :)
Oct 25 16:20:23 <gregdek> Fedora dudes?
Oct 25 16:20:29 <f13> Jesse Keating - Fedora Project Release Engineer and general Get Stuff Done kind of guy.
Oct 25 16:20:58 <notting> Bill Nottingham, Red Hat/Fedora generalist
Oct 25 16:21:00 <dgilmore> Im Dennis Gilmore, Im one of teh community members for Fedora Project that help maintain Fedora's build system
Oct 25 16:21:17 <_bernie> I'm Bernardo Innocenti, full time volunteer developer at OLPC
Oct 25 16:21:38 <c_scott> wow, it would be great to get all you people in a room together somewhere. but i guess this will have to do.
Oct 25 16:21:43 <gregdek> :)
Oct 25 16:21:46 <m_stone> Anyone else want to introduce themselves?
Oct 25 16:21:47 <gregdek> Small steps. :)
Oct 25 16:21:56 <_bernie> mostly in X11, localization and platform things
Oct 25 16:22:01 <m_stone> (If not, I propose that we agree to an agenda next)
Oct 25 16:22:05 <mbonnet> Mike Bonnet, Red Hat release engineer and one of the developers/maintainers of the Koji build system used by Fedora
Oct 25 16:22:59 <gregdek> m_stone: So my high-level agenda would be, "what acute problems do you guys have, and how can Fedora help with advice/resources/etc.?"
Oct 25 16:23:04 <m_stone> Okay.
Oct 25 16:23:06 <c_scott> ping now, or forever hold your introductions...
Oct 25 16:23:11 <m_stone> Could you write that on http://wiki.laptop.org/go/Build_system_meeting
Oct 25 16:23:12 <m_stone> ?
Oct 25 16:23:19 <_bernie> I'm putting the list from my e-mail on our wiki so we can all look at it and edit it
Oct 25 16:23:33 * f13 reads wiki
Oct 25 16:23:41 <f13> oh, guess I'll wait.
Oct 25 16:23:44 <m_stone> (I desperately want to write things down as we go so that we're able to explain our conclusions afterward)
Oct 25 16:23:50 <gregdek> m_stone: Want me to edit?
Oct 25 16:23:54 <m_stone> gregdek: please.
Oct 25 16:23:59 <c_scott> I'd like to hear the fedora guys give a quick talk about their community and build process, and what they feel are its strengths and weaknesses.
Oct 25 16:24:29 <_bernie> m_stone: just added a link to the requirements
Oct 25 16:24:36 <m_stone> _bernie: thanks.
Oct 25 16:24:46 <dgilmore> f13/mbonnet: want to do that ?
Oct 25 16:25:00 <gregdek> f13: I vote you. :)
Oct 25 16:25:05 <f13> heh, sure.
Oct 25 16:25:10 <_bernie> jg: you may want to augment them with what we discussed earlier (legal issues, scalability issues, burden for outside contributors, etc.)
Oct 25 16:25:21 <m_stone> _bernie: after we listen to f13.
Oct 25 16:25:23 <c_scott> especially your goals starting out, and perhaps what your goals now are (if they've changed over time)
Oct 25 16:25:26 <f13> The basic rundown is this. We have a public CVS tree.
Oct 25 16:25:33 <spevack> _bernie: a lot of those issues sound very familiar to our Fedora community building experiences :)
Oct 25 16:25:36 <f13> People gain write access to this tree by creating accounts in our Fedora Account System.
Oct 25 16:25:37 <_bernie> m_stone: (on the wiki)
Oct 25 16:25:59 <f13> we have a basic "sponsor" system by which somebody who is of "sponsor" level agrees to become responsible for you as a member.
Oct 25 16:26:27 <f13> Our CVS system has ACLs on it such that a maintainer can allow all authed users write access, individual users, or no users but themselves.
Oct 25 16:26:43 <jg> OK, concrete example: Etoys developer wanted to become fedora developer. He was concerned/upset about the legal agreement involved. I read it: he had good reason for concern: delay of 10 days while lawyers dealt with. ***We don't have time for 10 day delays right now****
Oct 25 16:26:48 <f13> Anybody who's sponsored in the account system has rights to build packages.
Oct 25 16:26:49 <m_stone> jg: please wait.
Oct 25 16:27:08 <f13> regardless of commit access
Oct 25 16:27:12 * olpc (n=olpc@BSN-95-201-206.dial-up.dsl.siol.net) has joined #olpc-meeting
Oct 25 16:27:27 <f13> We have a branching system where we branch the source control for each release, and now for subprojects like OLPC as needed
Oct 25 16:27:42 <f13> and we have a fine grained enough ACL system that there can be different rights on the branches
Oct 25 16:27:48 * olpc is now known as marco
Oct 25 16:28:20 <f13> The buildsystem itself could be a longer discussion
Oct 25 16:28:40 <f13> but the basic rundown is that it uses a database to "tag" builds of rpms for collections (buckets if you will)
Oct 25 16:28:59 <f13> There is the concept of inheritance so that you can easily bootstrap a new bucket without rebuilding everything.
Oct 25 16:29:21 <mbonnet> but the salient points of the build system are building directly from CVS tags, pristine buildroots for every build, tracking of build dependencies and buildroot contents, and tracking of build output
Oct 25 16:29:26 <f13> The buildsystem uses the contents of these buckets (and buckets it they inherit from) to populate "buildroots" to build packages in.
Oct 25 16:29:49 <f13> For OLPC we are using a convenient side effect of the buildsystem
Oct 25 16:30:12 <f13> in that the package repository the buildsystem creates to populate buildroots is suitable as a package repository for OLPC to create system images from.
Oct 25 16:30:31 <f13> These repos are created on demand, whenever an action happens that would lead to a change in the package set visible in the bucket.
Oct 25 16:30:37 <_bernie> For people who have never seen it before, the web interface to Koji looks like this: http://koji.fedoraproject.org/
Oct 25 16:30:55 <f13> (howver if there is already a repo creation in progress, then another repocreation will be delayed until the first one finishes. Only one at a time will run)
Oct 25 16:31:18 <f13> That's probably a good overview an I could continue on to the pain points if nobody objects.
Oct 25 16:31:46 <gregdek> Does everyone get it so far?
Oct 25 16:31:49 * _sj_ (n=sj_@wikipedia/sj) has joined #olpc-meeting
Oct 25 16:32:15 <f13> I'll be happy to clarify any part of it later too, as to not take up meeting time.
Oct 25 16:32:19 <c_scott> i'm not surei understand the 'side effect' but
Oct 25 16:32:23 <c_scott> s/but/bit/
Oct 25 16:32:32 <f13> c_scott: Our buildsystem uses standard components to work from
Oct 25 16:32:38 <_bernie> The web interface to the ACL system looks like this: https://admin.fedoraproject.org/pkgdb/collections/id/9
Oct 25 16:32:42 <c_scott> how is olpc's final build different from red hat's?
Oct 25 16:32:53 <f13> c_scott: yum, mock, rpm, etc.. We create yum repos that mock uses to create a fakeroot build environment.
Oct 25 16:33:17 <f13> c_scott: since these are standard tools, the OLPC image creation tool makes use of yum repos too. So the yum repo that the buildsystem uses can be the /same/ repo that the image creation tool uses.
Oct 25 16:33:33 <c_scott> how does red hat create it's public yum repos then?
Oct 25 16:33:37 <c_scott> s/it's/its/
Oct 25 16:33:43 <f13> c_scott: OLPC's build is typically the result of installing rpms into a fake root, stripping things out, and turning it into a file image.
Oct 25 16:34:05 <f13> c_scott: Fedora has to go through a few more steps due to shipping on multiple arches.
Oct 25 16:34:23 <f13> c_scott: that and our end product is not a disk image (rather not only a disk image), but also an installable media that has the raw rpms on it.
Oct 25 16:34:46 <c_scott> f13: could you elaborate? what additional steps are needed? why can't you run a disk image-creator based on an RPM repository, like we do?
Oct 25 16:35:03 <f13> c_scott: we have to pull packages out of the build system and set them up on a file system in a certain layout, do some post-processing to make it multilib for some arches (IE allow i386 content in an x86_64 tree)
Oct 25 16:35:25 <f13> c_scott: and then run anacona tools (or wrappers of such) to make the tree "installable".
Oct 25 16:35:37 <f13> c_scott: we can, for the disk images we create (IE our Live images)
Oct 25 16:36:04 <c_scott> ok, i'm still not seeing a fundamental difference, other than the multilib stuff.
Oct 25 16:36:11 <f13> c_scott: we have two different products really that we create. One is live images which parallel very closely with you, and the other is traditional "choose your own adventure" install cds and trees.
Oct 25 16:36:31 <f13> c_scott: for the live images the multilib is pretty much the only fundamental difference.
Oct 25 16:36:55 <f13> c_scott: We make our live images multilib so that they can run 32bit software on x86_64
Oct 25 16:36:56 <_bernie> c_scott: we make "live" images, they make "installation" images. there's an additional intermediate step.
Oct 25 16:36:59 <f13> (and ppc, and...)
Oct 25 16:37:11 * marcopg has quit (Connection timed out)
Oct 25 16:37:18 <f13> for our other target, we never really 'unpack' the rpms like we do for live images.
Oct 25 16:37:43 <f13> instead we present the user with a list of software they /could/ install. They pick and choose what they want, and /then/ they are unpacked on the user's filesystem.
Oct 25 16:37:57 <f13> this is more inline with how an OSX or windows install goes.
Oct 25 16:38:09 <c_scott> my question was why the "installation images" can't be built from an RPM repository -- why the RPM respository which we use "as a side effect" isn't everything that anyone would need.
Oct 25 16:38:09 <f13> (and how RHEL works as there currently are no Live images for RHEL)
Oct 25 16:38:25 <c_scott> s/anyone/any build process/
Oct 25 16:38:37 <f13> c_scott: OLPC's needs are far less than Fedoras, and thus it's suitable for OLPC to use the same yum repository that koji uses for buildroots.
Oct 25 16:39:01 <f13> c_scott: maybe it would help if I listed the difference between a koji buildroot repo and say the Fedora rawhide repo.
Oct 25 16:39:10 <c_scott> yes, i think that would help.
Oct 25 16:39:17 <_bernie> yes, please
Oct 25 16:39:51 <f13> c_scott: Koji's repos use unsigned rpms, are arch specific (no multilib), there are no source rpm repos, nor debuginfo package repos.
Oct 25 16:40:36 <c_scott> ah, yes. these were exactly (some of) the problems we need to fix for OLPC
Oct 25 16:40:50 <c_scott> so apparently the "side effect" isn't actually quite good enough for us.
Oct 25 16:40:58 <f13> in contrast, rawhide repos will use the "best" signed copy of an rpm it finds in koji, enable multilib (mix of i386/x86_64, ppc/ppc64), create sourcerpm repos, debuginfo package repos, expload release notes content in the tree, and a few other things.
Oct 25 16:41:13 <f13> c_scott: that's good to know, as I wasn't aware of this until now.
Oct 25 16:41:28 <c_scott> "expload release notes content" ?
Oct 25 16:41:57 <f13> c_scott: back when OLPC was based on FC6, I used to make more "rawhide" like repos for OLPC to use. However when you rebased to newer fedora it was thought that hte koji repos would be better because they happened automatically without any extra work.
Oct 25 16:42:10 <mbonnet> c_scott: extract the release notes from the fedora-release-notes RPM so they can but put onto the installation media
Oct 25 16:42:18 <f13> c_scott: if you look at a rawhide tree you'll see html content in the root of it. These are our release notes.
Oct 25 16:42:33 <f13> c_scott: also copies of the GPG keys we sign packages with
Oct 25 16:43:39 <f13> c_scott: if what koji produces for it's own needs is no longer suitable for your needs we really should outline what your needs are and find a better solution.
Oct 25 16:44:21 <c_scott> f13: ah, we need better release notes, too.
Oct 25 16:44:59 <c_scott> f13: in broad strokes: koji's *current* setup is unsuitable for a number of reasons (we'll get into details later)...
Oct 25 16:45:00 <_bernie> For everybody who's not familiar with it, these are the yum repositories that "come out" of koji and feed the pilgrim stage:
Oct 25 16:45:02 <_bernie> http://koji.fedoraproject.org/static-repos/
Oct 25 16:45:12 <c_scott> f13: it's not clear the *koji itself* is unsuitable, just how we are employing it
Oct 25 16:45:14 <mbonnet> I think it's important to note that the original source of the packages that go into Fedora and OLPC are both yum repos, created by koji. Fedora just needs to jump through some extra hoops that OLPC doesn't, at the moment.
Oct 25 16:45:19 <f13> c_scott: nod.
Oct 25 16:45:35 <gregdek> Thank goodness we have some Koji experts here. :)
Oct 25 16:45:36 <c_scott> f13: however, it *may* be that koji isn't really a good fit for the decentralized distribution work we'd like to do. if so, we might discuss alternatives.
Oct 25 16:45:41 <f13> mbonnet: that's not entirely true, but close enough.
Oct 25 16:45:47 <mbonnet> f13: details...
Oct 25 16:46:37 <f13> mbonnet: the tool we use, mash, doesn't use the koji build repos for information. It queries koji directly for package build IDs and then grabs the packages from the file store and creates the repo that way.
Oct 25 16:47:04 <f13> Are people ready to move on to the current pain points? (beyond just koji)?
Oct 25 16:47:14 <_bernie> This is the root of the rawhide tree f13 was talking about earlier: http://download.fedora.redhat.com/pub/fedora/linux/development/i386/os/
Oct 25 16:47:15 <c_scott> from previous discussions, we've established that koji is a very sophisticated pieces of software that does many things well. there's always a developer impulse to simplify something complex and replace it with something simpler, but often you just end up reinventing the wheel as you realize that all that complexity really *was* necessary.
Oct 25 16:48:13 <f13> indeed.
Oct 25 16:48:27 <f13> It's also worth noting that Koji is the product of rewriting something internal to Red Hat to be simpler (:
Oct 25 16:49:05 <c_scott> f13: i'd be very interested in learning more about mash.
Oct 25 16:49:17 <f13> While it works well for our uses, we certainly understand how it may not initially work well for other uses, but we have a strong desire to continue to use one tool and adjust it as necessary rather than a lot of code duplication.
Oct 25 16:49:32 <f13> c_scott: would you like me to talk about it now, or outside the meeting?
Oct 25 16:49:43 <f13> c_scott: or rather notting since he wrote it
Oct 25 16:49:50 <c_scott> f13: since OLPC has only forked about 40-some-odd packages right now; even if we created a brand new build system we'd like to be able to integrate with koji for the packages we don't fork.
Oct 25 16:50:11 <notting> c_scott: in what context? it has two main uses: 1) pulling packages from a tag into a repo (simple) 2) solve multilib repositories (paaaaaaaaaaaain)
Oct 25 16:50:35 <f13> c_scott: if that's the road we end up going down I'm confident we can still find ways to tie back into Koji and/or koji outputs.
Oct 25 16:50:35 <c_scott> so learning more about mash might help us figure out how we can build the new pieces we might need while still integrating with koji for the rest.
Oct 25 16:50:52 <f13> sure, lets give notting a moment to talk about the easy part of mash.
Oct 25 16:51:06 <f13> since, correct me if I'm wrong, OLPC is not a multilib platform, nor will it be for the forseeable future.
Oct 25 16:51:18 <f13> especially not in the way that Fedora / RHEL is multilib.
Oct 25 16:51:55 <notting> i'd hope not
Oct 25 16:52:08 * f13 too
Oct 25 16:52:30 <f13> since woodhouse is involved, I'd be reasonably certain the multilib insanity is kept well away from OLPC
Oct 25 16:52:33 <_bernie> c_scott: 46 now: https://admin.fedoraproject.org/pkgdb/collections/id/9
Oct 25 16:53:09 <_bernie> f13: pheeewww
Oct 25 16:53:11 <notting> so, the simple part of mash ; it runs on a simple config file that lists a koji server to contact, a koji tag to pull from, and whether or not to do inheritance from parents of that tag
Oct 25 16:53:40 <notting> it then contacts the koji server to get a list of all relevant builds. it then sorts them into trees by architecture, and makes repositories from them
Oct 25 16:53:46 <dgilmore> notting: which gets srpms and debug rpms also
Oct 25 16:53:49 <notting> right
Oct 25 16:53:58 <f13> in the context of OLPC that would mean "publis 46 packages, or publish the entirety of Fedora, and replace 46 packages with OLPC versions"
Oct 25 16:54:06 <c_scott> could we do something like this to get the srpms and debug rpms for the olpc-2 branch?
Oct 25 16:54:09 <notting> its big issue in this usage is that it requires access to the same filesystem that koji writes its packages to. it does *not* download the builds
Oct 25 16:54:20 <c_scott> ah.
Oct 25 16:54:34 <f13> this is good though
Oct 25 16:54:41 <_bernie> notting: it's not yum based, like pilgrim is?
Oct 25 16:54:43 <f13> as it makes use of hardlinks so there is very little duplication of data
Oct 25 16:55:04 <dgilmore> c_scott: you could
Oct 25 16:55:09 <f13> _bernie: pilgrim assumes a repo of packages exists. some tool somewhere has to make that initial repo, mash is such a tool.
Oct 25 16:55:15 <notting> _bernie: that part isn't. the part that does the multilib depsolving is
Oct 25 16:55:22 <_bernie> I see
Oct 25 16:55:36 <c_scott> one option for us has been to run our own instance of koji, and delegate to upstream koji for the packages we haven't forked. getting access to the upstream koji's products is part of that particular solution.
Oct 25 16:55:44 <f13> c_scott: and yes, mash could be used to produce such content from the olpc-2 branch.
Oct 25 16:55:50 <c_scott> it's not clear to be that koji inherently has any support for such delegation?
Oct 25 16:55:54 <c_scott> s/be/me/
Oct 25 16:56:00 <f13> c_scott: not really, or not well.
Oct 25 16:56:17 <f13> c_scott: we're developing some of that for other needs in Fedora
Oct 25 16:56:26 <mbonnet> c_scott: that's similar, though not exactly the same, as what the secondary arch guys are doing
Oct 25 16:56:38 <dgilmore> c_scott: there is not that yet but simmiliar functionality is being worked on
Oct 25 16:56:39 <f13> c_scott: like secondary Fedora arches (ia64, s390x, sparc, arm, alpha)
Oct 25 16:56:44 <c_scott> ok, this might be a good time to turn to our itemized list of "build system requirements" and see if we can discuss particular issues.
Oct 25 16:57:09 <f13> if we do that, I'd like to get a bit of glossary created
Oct 25 16:57:15 <c_scott> (we've though about treating the XO as a special 'arch' in the distro -- that would let us use processor-specific optimizations, for instance)
Oct 25 16:57:17 <_bernie> f13: good to know. this would address a major scalability concern of a too centralized process.
Oct 25 16:57:20 <m_stone> okay. What are you thinking of?
Oct 25 16:57:46 <m_stone> (re "glossary")
Oct 25 16:57:47 <c_scott> m_stone: who were you responding to?
Oct 25 16:57:51 <c_scott> oh ah
Oct 25 16:59:20 <_bernie> c_scott: would you like to pick from here?
Oct 25 16:59:20 <_bernie> http://wiki.laptop.org/go/Build_system_requirements
Oct 25 16:59:21 <f13> The glossary we mostly use is such: "Build System" The job scheduling and rpm building software (koji). "SCM" The source control where build instructions and sources are pulled from (CVS), "Compose tools" Things that take output from the buildsystem (rpms, srpms) and 'compose' them into various formats (mash, [pungi, livecd-tools, pilgrim])
Oct 25 16:59:55 <c_scott> ah, that's a good start. the terms are often confusing.
Oct 25 16:59:57 <f13> "build system" often gets overloaded and can sometimes be used to refer to the entire stack.
Oct 25 17:00:08 <_bernie> c_scott: so what we call "build system" here should have been the "compose tool".
Oct 25 17:00:15 <c_scott> i'd like to separate out 'pilgrim' from the rest of the 'compose tools'
Oct 25 17:00:16 <f13> I don't have a good term for the entire stack, nor a good term for just the 'koji' part of the system.
Oct 25 17:00:41 <c_scott> since I think we want things like changelog notification (say) which are compose tools that operate on metadata, mostly independent of the actual image-creation process
Oct 25 17:00:49 >f13< distro assembly line?
Oct 25 17:00:51 <f13> c_scott: sure. I listed it in brackets with pungi and livecd-tools because they all operate similarly on they need a yum repo to point to.
Oct 25 17:00:59 <f13> whereas mash needs a koji to point to.
Oct 25 17:01:12 <c_scott> let's just call 'pilgrim' by its name, and accept that we're using it as a generic term to relate to all such image-creation tools
Oct 25 17:01:19 <f13> can do
Oct 25 17:01:25 <c_scott> let's reserve 'compose tools' for other tools that operate on metadata or repositories
Oct 25 17:01:48 <m_stone> f13: we have a nice picture on our whiteboard here that I want to reproduce for you.
Oct 25 17:02:04 <f13> hrm, maybe I should have come in...
Oct 25 17:02:11 <c_scott> also, i'd like to distinguish 'koji' from -- let's call it 'mock' -- that is, the management pieces from the buildroot builder building piece
Oct 25 17:02:14 <m_stone> f13: it's an easy picture to do in ascii.
Oct 25 17:02:21 <m_stone> (just a flow chart)
Oct 25 17:02:30 <f13> oh yeah, I live in the Boston area and have been to your offices before, so I can always stop by for a more detailed brain dump complete with whiteboard fun.
Oct 25 17:03:24 <f13> c_scott: ok, that's getting into harder to grasp areas. Currently there is no koji without mock.
Oct 25 17:03:33 <c_scott> right, but we might want mock w/o koji. *might*
Oct 25 17:03:53 <c_scott> or use koji, but have our own local buildservice.
Oct 25 17:04:17 <c_scott> in any case, if we have terms to distinguish the two, we can discuss the possibilities.
Oct 25 17:04:20 <dgilmore> c_scott: you can do that. there is no other tool out there that will let you know what exactly was in the build root and let you be able to 100% reproduce a build
Oct 25 17:04:41 <m_stone> src -> src pkgs ---> bin pkgs ---> FS assembly --> img
Oct 25 17:04:41 <f13> c_scott: sure, we can refer to them as such for conversation.
Oct 25 17:04:47 * _mostro has quit ("Leaving")
Oct 25 17:04:50 <c_scott> dgilmore: i'd be cautious of your use of absolutes
Oct 25 17:05:02 <m_stone> with another arrow going from "src" to "bin pkgs"
Oct 25 17:05:05 <f13> m_stone: what does "FS" refer to?
Oct 25 17:05:09 <m_stone> file-system.
Oct 25 17:05:13 <f13> I see.
Oct 25 17:05:33 <m_stone> there's a few more items going into file-system assembly that we desperately want to remove.
Oct 25 17:05:38 <c_scott> i think our current glossary is adequate for our present needs.
Oct 25 17:05:44 <c_scott> let's start with the first item on http://wiki.laptop.org/go/Build_system_requirements
Oct 25 17:06:03 <c_scott> which is to make it clear that we don't want to shift to a "non red hat" way of doing things
Oct 25 17:06:18 <c_scott> ie, we're based on fedora core, and continue to plan to fork as little as possible.
Oct 25 17:06:27 <f13> a big +1 from me on that.
Oct 25 17:06:42 <c_scott> so our first concern is that we interoperate well and can both incorporate upstream changes and contribute our patches upstream.
Oct 25 17:07:15 <f13> c_scott: upstream as in Fedora, or upstream as in the actual upstream of the particular peice of software in question?
Oct 25 17:07:16 <dgilmore> c_scott: that is very good to hear
Oct 25 17:07:20 <c_scott> we've already mentioned this a little before, but even if we reinvent our own build system from scratch, we'll want to interoperate with koji closely.
Oct 25 17:07:42 <c_scott> f13: both, probably. i meant fedora upstream when i wrote it.
Oct 25 17:08:07 <dgilmore> patches can be pushed through fedora to upstream projects
Oct 25 17:08:18 * dgilmore has done that with sparc patches
Oct 25 17:08:23 <f13> or just directly upstream
Oct 25 17:08:28 <dgilmore> or that
Oct 25 17:08:33 <f13> but that's in the weeds.
Oct 25 17:08:52 <f13> I think we're with you so far on your requirements (and in agreement)
Oct 25 17:09:00 <c_scott> pushing through redhat/fedora allows us to leverage fedora's resources to communicate with upstream
Oct 25 17:09:11 <c_scott> instead of barraging upstream with lots of individual inquiries/pathces.
Oct 25 17:09:29 * unmadindu has quit (Read error: 110 (Connection timed out))
Oct 25 17:09:44 <c_scott> actually, our second requirement on the list is more of the same: " easy way to resynch with upstream for the packages we need to fork [cscott, bernie]"
Oct 25 17:09:54 <_bernie> yes
Oct 25 17:10:38 <c_scott> let's keep going and get to the places where koji might fail us, and then return to requirements 1 and 2 if/when we discuss alternatives to koji.,
Oct 25 17:10:43 <dgilmore> c_scott: by that do you mean getting what you change in the fedora package and reverting to using it rather than your own build?
Oct 25 17:10:44 <f13> nod.
Oct 25 17:11:06 <_bernie> preliminary picture: http://wiki.laptop.org/go/Build_system_meeting#Picture_of_the_distro_assembly_line
Oct 25 17:11:07 <c_scott> dgilmore: yes that, and also pulling in security patches from upstream even if we've got our own forked copy.
Oct 25 17:11:43 <c_scott> so two way synchronization, even if there are OLPC-specific changes which aren't appropriate for upstream.
Oct 25 17:12:03 <dgilmore> c_scott: ok
Oct 25 17:12:14 <c_scott> ok, item 3 is stable builds. the only way that stock koji fails us here is in allowing OLPC-specific configurations.
Oct 25 17:12:41 <c_scott> if/as the OLPC-specific configuration diverges from FC7, we have a greater risk when using 'inheritance' to assume that binaries are sharable.
Oct 25 17:12:46 <f13> can you expand upon that?
Oct 25 17:12:56 <f13> what would be an example of an 'OLPC-specific configuration'
Oct 25 17:14:11 <c_scott> so if, say, we have our own version of (say) gstreamer which allowed Fooness, we're at risk of having FC7 packages inherently depend on non-Fooness, and we might not notice that we need to fork those packages.
Oct 25 17:14:43 <c_scott> At the moment, for example, we're FC7-based but using FC8 glibc in order to get ethiopian support. Will that break other packages? Who knows?
Oct 25 17:15:08 <f13> Ok, I understand the problem, what would be a solution that koji doesn't allow?
Oct 25 17:15:29 <_bernie> c_scott: apart from breaking things, we have no way to merge back this change at this time
Oct 25 17:15:48 <c_scott> i'm saying that our current use of koji doesn't handle this problem (read, protect us from this problem) very well. it may be possible that we could use koji differently.
Oct 25 17:15:52 <_bernie> c_scott: glibc is not branched for OLPC-2 and so far none of us can add branches (but this is easy to solve)
Oct 25 17:15:58 <c_scott> for example, maybe the XO should be a separate arch (implicitly forking all builds)
Oct 25 17:16:10 <gregdek> Hm.
Oct 25 17:16:38 <gregdek> That would also allow us an easy way of pushing XO builds to a different set of build hosts, maybe?
Oct 25 17:16:49 <f13> gregdek: nonissue
Oct 25 17:16:53 <_bernie> c_scott: if it's just for geode optimizations, maybe that can be achieved by just forking the packages we care to optimize
Oct 25 17:16:55 * gregdek shuts up. :)
Oct 25 17:17:04 <_bernie> c_scott: or do like they do with subarches: athlon, i686...
Oct 25 17:17:10 <f13> gregdek: Koji has a mechinism of "channels", so we can assign builders to specific channels, and jobs to specific channels.
Oct 25 17:17:19 <_bernie> gregdek: oops, sorry.
Oct 25 17:17:29 <gregdek> f13: You have progressed far from the simple days that I remember. ;)
Oct 25 17:17:32 <m_stone> _bernie: it is *not* easy to do the education required to learn how to use all of this.
Oct 25 17:17:35 <f13> _bernie: you bring up a good point.
Oct 25 17:17:37 <c_scott> i'm more worried about accidentally breaking packages because we're forking *pieces* of the distro -- the optimization opportunities are just a "would be nice" thing.
Oct 25 17:18:09 <f13> so it's almost a QA point that is missing.
Oct 25 17:18:16 <_bernie> m_stone: granted, but whatever we come up with as a replacement, it may be even harder to use and certainly less documented :-)
Oct 25 17:18:36 <f13> if you're more in tuned with Fedora you can gain the QA happening on Fedora bits, but as you venture away you have a QA point that you have to worry about with far less visiblity
Oct 25 17:18:37 <_bernie> f13: some time ago I convinced someone on #fedora-devel to add "geode" to the list of arches in Makefile.common
Oct 25 17:18:44 <f13> nod
Oct 25 17:18:47 <_bernie> f13: now I can "make geode" in my cvs trees
Oct 25 17:18:53 <f13> so just for everybody to be on the same page...
Oct 25 17:18:53 <c_scott> f13: almost -- but it could also be solved technically by ensuring that all our packages were compiled in an appropriate olpc-2 environment
Oct 25 17:18:59 <_bernie> f13: something else is missing in rpm macros, etc.
Oct 25 17:19:13 <f13> There is a nuance(sp?) of sub-arches in how koji works.
Oct 25 17:19:29 <_bernie> but geode support is as trivial as grepping for "athlon" in the redhat tools and adding "geode" besides it.
Oct 25 17:19:42 <f13> A "sub-arch" is like 'i386, i486, athlon, i686' are all subarches of the i386 (x86) arch.
Oct 25 17:19:49 <f13> in Koji land we just refer to it as 'i386'
Oct 25 17:20:09 <f13> but koji /can/ actually build a package for a specific sub-arch, like glibc. We produce glibc builds for i386 and i686
Oct 25 17:20:41 <f13> the idea is that we could treat 'geode' as yet another subarch if x86 and we can build some of these packages for that sub-arch as well.
Oct 25 17:20:58 <c_scott> let's keep going through our list, and then return to discuss specific solutions when we're done. that way we don't solve subproblems before we know the whole picture. does everyone understand the issue embodied in this particular bullet point?
Oct 25 17:21:08 <f13> this is less work that doing olpc as a full arch on its own, it's more opt-in for specific packages.
Oct 25 17:21:13 <f13> c_scott: absolutely.
Oct 25 17:21:17 * djbclark has quit (Remote closed the connection)
Oct 25 17:21:22 <c_scott> subarches don't by themselves solve the "FC8 glibc" problem --- but moving to FC8 would ;-)
Oct 25 17:21:35 <c_scott> ok, "ability to fork and unfork packages for OLPC"
Oct 25 17:21:53 <jg> f13, _bernie: we will *not* be doing further optimizations of packages between now and FRS; any discussion of that is after our first deployment...
Oct 25 17:22:14 <_bernie> jg: yes, definitely
Oct 25 17:22:28 <_bernie> jg: I'm delaying Rob's glibc optimizations too for the same reason
Oct 25 17:22:33 <f13> c_scott: yes, this was one of the pain points I was going to list.
Oct 25 17:22:35 <c_scott> i think the issue here ("fork and unfork") is primarily ACLs. we'd like our own little sandbox; we should be able to do OLPC-specific hacks w/o bothering the rest of the fedora builds.
Oct 25 17:22:35 <dgilmore> c_scott: you can mark packages to built built for additional archs
Oct 25 17:23:04 <gregdek> I think the "little sandbox" discussion is an important one, beyond technical reasons.
Oct 25 17:23:06 <f13> c_scott: right. Currently much of our adminstration is an all or nothing mind set.
Oct 25 17:23:09 <c_scott> part of the problem is that it's easy for us to modify packages once we've forked them (and gotten someone assigned ACLs), but it's initially very hard to fork a new package.
Oct 25 17:23:16 <f13> you can either create branches for everything, or you can't create branches at all.
Oct 25 17:23:30 <c_scott> so our pilgrim script has gotten littered with patches which get applied at build time, because that was easier than dealing with the fork process.
Oct 25 17:23:55 <f13> So it may not be easy to turn over the ability to create /just/ olpc-2 branches, but at the same time, we're a trusting folk and we wouldn't necessarily object to granting somebody from your org the ability to create branches.
Oct 25 17:24:07 <c_scott> yes, i've gotten ACLs for all of 'initscripts', for example, even though it's only the OLPC-2 branch i'm interested in.
Oct 25 17:24:08 <f13> this can give you a much faster turn around time.
Oct 25 17:24:24 <dgilmore> f13: perhaps we could look at some finer grained cvs acls to allow people to branch only to OLPC-2
Oct 25 17:24:25 >c_scott< (for the record, we've been using F8 glibc already for a week or so)
Oct 25 17:24:32 <f13> so that you can fork new packages on demand and not wait for one of our admins to process the rest of the pending branch requests.
Oct 25 17:24:37 <c_scott> f13: yes, that might be a solution. everyone on board with this bullet? on to the next?
Oct 25 17:24:50 <f13> dgilmore: we could look at them longer term absolutely. I just want to make sure that it's know that this isn't a blocker.
Oct 25 17:25:05 <dgilmore> f13: agreed
Oct 25 17:25:24 <c_scott> as long as you trust us sufficiently it's not a blocker. ;-)
Oct 25 17:25:37 <m_stone> c_scott: not yet.
Oct 25 17:25:52 <_bernie> Improved picture of our current build system: http://wiki.laptop.org/go/Build_system_meeting#Picture_of_the_distro_assembly_line
Oct 25 17:25:52 <c_scott> but i would like to see a longer-term plan for fine-grained ACLs ---
Oct 25 17:26:02 <f13> c_scott: right. There is a trust issue there, but it's worth noting that most our CVS admins are not Red Hat employees.
Oct 25 17:26:24 <m_stone> c_scott: even when RH folk were working with us every day, right next door, we *still* were not able to get beyond the human bottlenecks introduced by having K people with admin privileges for small values of K.
Oct 25 17:26:48 <f13> m_stone: one of the problems there was the RH people with you weren't the ones with admin privs.
Oct 25 17:26:54 <gregdek> Heh.
Oct 25 17:27:01 <jg> we rest our case....
Oct 25 17:27:01 <c_scott> yes, we can partially solve this by giving an additional K local people access, but finer-grained ACLs are necessary in the medium-term.
Oct 25 17:27:05 <f13> m_stone: /and/ we were in the middle of trying to merge core and Extras, which brought /all/ kinds of admin headaches.
Oct 25 17:27:11 <m_stone> so while the bottleneck might be slightly wider if we have someone local who can do that package creation - it's still darn narrow.
Oct 25 17:27:15 <f13> just bad timing all around.
Oct 25 17:27:19 <f13> We're in a much better place now.
Oct 25 17:27:28 <m_stone> f13: can you say more here?
Oct 25 17:27:43 <f13> m_stone: sure, it's not private info at all.
Oct 25 17:27:43 <m_stone> this is probably my biggest concern.
Oct 25 17:28:02 <m_stone> f13: by "say more", I'm specifically interested in knowing several things.
Oct 25 17:28:07 <f13> We were trying to merge the administration and management of two different software projects together.
Oct 25 17:28:27 <m_stone> 1) what your situation was. 2) what it is now. 3) will your situation ever recur. 4) what can we do to remove said bottleneck.
Oct 25 17:28:34 <dgilmore> c_scott: im not an RH employee and i have access to pretty much everything in fedora
Oct 25 17:28:36 <f13> Extras was self governing, completely external to Red Hat. Core was internal to Red Hat and had a very strict setup.
Oct 25 17:29:15 <f13> combining the two of them was a very long and rough process, trying to find happy mediums between the two, identify who was going to control what and where and how to deciminate authority
Oct 25 17:29:49 <f13> but now that we've accomplished this, we have better tools in place for granting people access to adminstrate things like the CVS system or Koji
Oct 25 17:30:13 <f13> and we have history of non Red Hat people doing the bulk of this work and we (the RH employees) are more comfortable with releasing control.
Oct 25 17:30:35 <f13> I don't necessarily envision this situation happening again, as we no longer have a secondary package repository
Oct 25 17:30:38 <gregdek> (Thank God. It was a long and hard fight.)
Oct 25 17:30:50 <c_scott> dgilmore: yes, but say that you gave me access to everything in Fedora. I still need to delegate most of the work to other people, and i might not necessarily trust (say) _bernie with anything other than X. I might not even trust _bernie with anything other than OLPC X. I might not even know _bernie very well. If I don't have fine-grained ACLs, I end up having to do all the admin work myself.
Oct 25 17:31:06 <f13> honestly getting the merger accomplished was one of the big reasons why I came to work at Red Hat. I knew I could get it to happen easier from the inside.
Oct 25 17:31:14 <gregdek> I think we agree that fine-grained ACLs are important.
Oct 25 17:31:16 <c_scott> If I didn't know bernie, giving him access to X would be unwise absent a way to ensure he only committed on the olpc-2 branch.
Oct 25 17:31:27 <f13> lets step back a bit
Oct 25 17:31:30 <gregdek> But until now, we haven't had a use case that *demanded* them like OLPC does.
Oct 25 17:31:36 <c_scott> ok, let's keep going with our list of concerns -- if we decide to run our own koji (for example) this problem might magically go away. i want to discuss all the concerns before we dive too deeply into possible solutions.
Oct 25 17:31:37 <_bernie> c_scott: afaik, they can give you access limited to branches
Oct 25 17:31:39 <f13> we're talking about two differert kinds of access.
Oct 25 17:31:44 <m_stone> c_scott: hang on a second here.
Oct 25 17:32:00 <m_stone> f13: indeed. I'm primarily concerned with the ability to introduce new packages to Fedora.
Oct 25 17:32:02 <f13> 1) admin rights to create package branches in CVS.
Oct 25 17:32:21 <f13> 2) commit access to pre-existing packages/branches in CVS
Oct 25 17:32:31 <f13> 1 is more the all or nothing scenario.
Oct 25 17:32:33 <m_stone> f13: and with "who do I have to know in order to work the system".
Oct 25 17:32:34 <_bernie> c_scott: see here: https://admin.fedoraproject.org/pkgdb/packages/name/initscripts
Oct 25 17:32:35 <dgilmore> c_scott: certainly was just illustration that its not some process tied to RH
Oct 25 17:32:36 <c_scott> m_stone: you're talking about the next point on the list.
Oct 25 17:32:40 <m_stone> c_scott: ah.
Oct 25 17:32:40 <_bernie> c_scott: I can only commit to OLPC-2
Oct 25 17:32:41 <f13> 2 we have much better fine grained access.
Oct 25 17:33:11 * djbclark (i=dclark@opensysadmin.com) has joined #olpc-meeting
Oct 25 17:33:14 <_bernie> m_stone: show us the specific bz :-)
Oct 25 17:33:15 <f13> m_stone: I have some solutions to that too, but lets wait until the next point.
Oct 25 17:33:20 <c_scott> ok, but if we decide to fork all packages at once so that (1) didn't matter any more, we'd be back to our first concerns -- tracking upstream changes.
Oct 25 17:33:31 * _sj_ has quit (Read error: 110 (Connection timed out))
Oct 25 17:33:41 <c_scott> ok. next point: ability to add/remove custom packages from the build without going through a lengthy review process
Oct 25 17:33:51 <f13> so to recap, we're welcome to adding (more) olpc people to the 1 group, the people who can do CVS admin tasks, like creating branches or preparing a new branch for package import.
Oct 25 17:33:59 * gregdek raises his hand.
Oct 25 17:34:15 <c_scott> ok, backing up to previous point.
Oct 25 17:34:17 <f13> c_scott: I have a point of contention here.
Oct 25 17:34:20 <m_stone> f13: to continue the recap - it's not clear that that solves our problems.
Oct 25 17:34:22 <f13> (holding it)
Oct 25 17:34:28 <m_stone> gregdek: your turn.
Oct 25 17:34:33 <f13> m_stone: it certainly solves one part of the problem.
Oct 25 17:34:58 <gregdek> So how much of the problem is just "it takes too long to get packages accepted into mainline Fedora"?
Oct 25 17:35:13 <m_stone> gregdek: It's more a communication problem.
Oct 25 17:35:25 <c_scott> f13: creating branches doesn't automatically give new developers the right to commit to that branch -- part of our issue has been quickly forking an arbitrary package, without having to separately negotiate (a) the cvs fork bits, and (b) the write access to repository bits.
Oct 25 17:35:27 <f13> I'd really like to run through a couple things here.
Oct 25 17:35:36 <f13> may I have the floor?
Oct 25 17:35:36 <m_stone> gregdek: see https://bugzilla.redhat.com/show_bug.cgi?id=250463 for a prime example.
Oct 25 17:35:41 <_bernie> gregdek: for me it's been easy because I had support of J5. But it still took a few days for a trivial package such as a font.
Oct 25 17:35:47 <gregdek> I'm happy to yield to f13...
Oct 25 17:35:49 * _sj_ (n=sj_@wikipedia/sj) has joined #olpc-meeting
Oct 25 17:35:52 <c_scott> let's give the floor to f13.
Oct 25 17:35:55 <m_stone> okay.
Oct 25 17:36:07 <f13> Our package entry work flow is indeed cumbersome.
Oct 25 17:36:19 <f13> And right now OLPC is completely dependant on the Fedora way of doing things, for better of for worse.
Oct 25 17:36:29 <f13> /and/ OLPC packages aren't necessarily treated any differently.
Oct 25 17:36:34 <c_scott> (not completely, but we'll discuss that later)
Oct 25 17:36:54 <f13> The lag comes in A) getting somebody to review / approve the package.
Oct 25 17:37:10 <f13> B) getting a CVS admin to prepare the CVS system for inital import and ACLs
Oct 25 17:37:27 <f13> and C) finally importing the package and building it.
Oct 25 17:37:39 <f13> This is entirely a people problem, not a process problem.
Oct 25 17:37:41 <c_scott> there's a legal issue as well
Oct 25 17:37:50 <f13> c_scott: oh right.
Oct 25 17:37:55 <f13> I'll visit that too
Oct 25 17:38:08 <c_scott> we might be thinking of different legal issues. let's be sure to make time for that. but finish your thoughts.
Oct 25 17:38:10 <f13> For /brand new/ packages to Fedora there is a legal requirement that they sign a Fedora CLA
Oct 25 17:38:19 <m_stone> f13: when you finish, I would like to offer my take on some difficulties that I see that you have not yet mentioned.
Oct 25 17:38:36 <f13> This si to protect Red Hat /Fedora's intrest as well as any consumer of Fedora.
Oct 25 17:38:51 <jg> f13: that agreement is fundamentally broken.
Oct 25 17:39:04 <jg> I explained how and why to spevack in a side conversation.
Oct 25 17:39:04 <f13> Because like it or not, there are legal requirements involved with producing software and content.
Oct 25 17:39:26 <f13> jg: the Fedora CLA itself may or may not be flawed. An agreement of /some kind/ is necessary.
Oct 25 17:39:26 <c_scott> f13: also, the agreement seems to expect that the *package maintainer* is separate from the *upstream author*. the package maintainer might expect to give away additional rights to the packaging, which upstream would like to reserve.
Oct 25 17:39:37 <jg> it isn't something we can solve in time for our first deployment.
Oct 25 17:39:41 <jg> so it is pretty moot.
Oct 25 17:39:45 <f13> jg: actually it's not.
Oct 25 17:39:48 <f13> please hear me out.
Oct 25 17:39:57 * m_stone seconds hearing out f13.
Oct 25 17:40:05 <jg> Let's not go into the details of its numerous bugs: we had about 3 pages of discussion.
Oct 25 17:40:16 <f13> Within the Fedora Account system we have a special group called "cla_done". This is a meta group.
Oct 25 17:40:31 <f13> it actually inherits people from cla_fedora, cla_redhat, cla_dell, and I think one other.
Oct 25 17:40:46 <f13> Each of /these/ subgroups actually have a different CLA that is signed.
Oct 25 17:41:00 <f13> All Fedora cares about si that the user eventually winds up in cla_done.
Oct 25 17:41:25 <f13> A cla_olpc group can be created that has an agreement in it that is agreeable and suitable for OLPC and for Red Hat / Fedora
Oct 25 17:41:27 <m_stone> (and that Fedora is able to legally distribute software)
Oct 25 17:41:28 <c_scott> I second jg in thinking that we can't get another CLA agreement approved by legal in time for this to matter.
Oct 25 17:41:46 <f13> it can be managed by somebody from OLPC as far as approving people's membership
Oct 25 17:42:26 <f13> I will conceed that I don't know how long it would take to accomplish this agreement, but it would most certainly be /far/ faster than trying to change the Fedora CLA that most people sign.
Oct 25 17:42:31 * gregdek has a question.
Oct 25 17:42:42 <m_stone> gregdek: please ask your question.
Oct 25 17:43:33 <jg> f13: I don't have time to go around asking anyone to sign any agreement at all between now and FRS; OLPC must take responsibility itself for what it does.
Oct 25 17:43:37 <gregdek> Is it necessary, in the *very* short term, for OLPC to necessarily contribute RPMs back into the Fedora universe? Or could OLPC carry forked RPMs for some period of time? I think the answer might have a bearing on short-term solutions.
Oct 25 17:43:54 <f13> gregdek: they already do
Oct 25 17:44:07 <gregdek> f13: Right.
Oct 25 17:44:14 <f13> jg: again, I feel that /OLPC/ can decide how to govern the population of cla_olpc.
Oct 25 17:44:34 <gregdek> So let me rephrase my question:
Oct 25 17:44:36 <f13> jg: the existance of somebody in cla_olpc would mean to Fedora that the person has agreed to the terms that it sets forth
Oct 25 17:45:05 <jg> f13: I can't, in the available time, give any assurances to anyone about whether an OLPC contributor is "legit".
Oct 25 17:45:15 <c_scott> gregdek: i agree. this is why i want to continue discussing our requirements list. i think that at the end of the day we're going to decide that at least some of OLPC's sources need to live on OLPC's servers. I want to figure out how to best do that.
Oct 25 17:45:17 <gregdek> If OLPC is not injecting content into Fedora, but instead using Fedora infrastructure and carrying forked code for now -- is there any reason for OLPC to sign the CLA *at all*?
Oct 25 17:45:20 <f13> jg: how OLPC gets people into that account is up to OLPC. it need not involve a gpg signed document, because if there is a dispute, it would be between that user and OLPC, not Fedora.
Oct 25 17:45:25 <jg> so I don't understand how Fedora could take that risk for Fedora.
Oct 25 17:45:50 <m_stone> jg: let's not deal with it now, please?
Oct 25 17:46:18 <m_stone> we can take f13 at his word that if the lawyers hashed it out, it would be technically straightforward to accomplish.
Oct 25 17:46:25 <f13> jg: so I'm talking /well/ beyond my "paygrade" as it were, but I envision that all Fedora cares about is that the user winds up in yoru CLA group. But I'm so far beyond what I do that I could be completely wrong.
Oct 25 17:46:47 <c_scott> f13: we heard completely different things from red hat management when we have previously discussed this
Oct 25 17:46:53 <c_scott> sarbanes-oxley was mentioned
Oct 25 17:46:55 <gregdek> What is the purpose of the Fedora CLA?
Oct 25 17:46:58 <f13> I hate that term.
Oct 25 17:47:06 <gregdek> Sarbox is the boogeyman when people have no answers and no time to find them.
Oct 25 17:47:11 <gregdek> So let's get real.
Oct 25 17:47:28 <gregdek> The purpose of the CLA is to protect Red Hat's ability to reship stuff in the Fedora universe without liability.
Oct 25 17:47:32 <c_scott> gregdek: i agree, i'm just saying that there is no guarantee that all of red hat/fedora will be as enlightened as f13
Oct 25 17:47:36 <gregdek> But the question, *in the near term*, is this:
Oct 25 17:47:48 <gregdek> Is OLPC stuff part of the Fedora universe?
Oct 25 17:47:49 <f13> gregdek: at the basic level I think it's to prevent Red Hat from being sued by somebody's employer for shipping works of that employee in an opensource product or worse in a RHEL release.
Oct 25 17:48:00 <gregdek> Or is OLPC simply using Fedora's infrastructure?
Oct 25 17:48:12 <gregdek> I think the difference is *absolutely key*.
Oct 25 17:48:24 <gregdek> And believe me: no one here has talked more with RH Legal than I have.
Oct 25 17:48:41 <f13> I'm also concerned that in the "short term" that everybody is concerned with, just how many brand new packagers you're expecting to fold into the system.
Oct 25 17:48:43 <c_scott> i think we would like to be more like ubuntu is to debian -- completely separate, but making a best effort to integrate and converge with upstream
Oct 25 17:49:00 <gregdek> c_scott: That may well be the right solution over time.
Oct 25 17:49:01 <notting> apologies. i have to leave now
Oct 25 17:49:05 <c_scott> we shouldn't have to wait for fedora's okay for anything, but we should make our best effort to eventually get that okay.
Oct 25 17:49:07 <m_stone> notting: thanks for coming.
Oct 25 17:49:10 <gregdek> Is that the right solution in the short term? The medium term?
Oct 25 17:49:15 * notting has quit ("Ex-Chat")
Oct 25 17:49:17 <_bernie> notting: thank you very much for your time
Oct 25 17:49:21 <_bernie> oops too late
Oct 25 17:49:22 <m_stone> c_scott: that's not so obvious to me.
Oct 25 17:49:32 <gregdek> c_scott: I'd like to make sure that we can support you *without* the requirements to get Fedora's okay.
Oct 25 17:49:43 <c_scott> i'm open to other suggestions, but i also like gregdek's take.
Oct 25 17:50:03 <gregdek> If we could treat the Fedora infrastructure as simply a service to OLPC, but the content is not Fedora content, then the CLA simply shouldn't matter.
Oct 25 17:50:04 <f13> jg: also, the usage of cla_olpc may only make sense for employees of olpc. Just connected a dot in my thought process.
Oct 25 17:50:17 <jg> yup.
Oct 25 17:50:24 <jg> there we have an agreement in place.
Oct 25 17:50:33 <c_scott> our development pace has been very fast, which has engendered a lot of bad will because fedora/red hat have been seen as a drag on our development. if we can get rid of the burden, i think our relationship will improve.
Oct 25 17:50:37 <jg> (or contractors of OLPC).
Oct 25 17:50:45 <gregdek> Abso-freaking-lutely.
Oct 25 17:51:03 <c_scott> f13, jg: yes, but _bernie (for example) is not an employee of OLPC.
Oct 25 17:51:08 <jg> true.
Oct 25 17:51:15 <gregdek> Fedora has great infrastructure and poor process. Why? Because the law *must* stand in the way of *Fedora* process.
Oct 25 17:51:18 <f13> I'm with gregdek here. I'm all for using the full Fedora when possible, and I want to make it as easy as possible /where I can/. But when you have to go around, we need to make it possible for you to go around too.
Oct 25 17:51:20 <m_stone> c_scott: basically, I'm open to playing by fedora's rules but I know that I was inordinately slowed down by an inability to understand what those rules were.
Oct 25 17:51:20 <gregdek> The same does *not* hold true for OLPC.
Oct 25 17:51:37 <gregdek> Which means...
Oct 25 17:51:40 <jg> but he has signed an NDA, and we could have him sign something else to cover this case.
Oct 25 17:52:02 <gregdek> ...that we should be able to give OLPC *all* the power, and *none* of the process overhead, if we structure the agreement correctly.
Oct 25 17:52:09 <f13> jg: right, and then you could safely add him to cla_olpc. But now we're beating a horse.
Oct 25 17:52:21 <c_scott> ok. can we briefly return to requirements?
Oct 25 17:52:23 <jg> let's let the horse rot.
Oct 25 17:52:23 <f13> So we had a nice tangent, I"d like to get back to what I was talking about
Oct 25 17:52:32 <m_stone> f13: please do!
Oct 25 17:52:38 <f13> So 1) new packagers need CLA, this we know is a concern.
Oct 25 17:53:04 <f13> then we had 2) review the package (and "sponsor" new packagers) We know this can take a while under current situations.
Oct 25 17:53:13 <f13> then 3) prepare the cvs system
Oct 25 17:53:19 <_bernie> m_stone: the documentation has somewhat improved recently
Oct 25 17:53:20 <f13> 4) import the package and build it.
Oct 25 17:53:20 <_bernie> http://fedoraproject.org/wiki/PackageMaintainers/Join
Oct 25 17:53:26 <_bernie> m_stone: ^^^ see above
Oct 25 17:53:35 <f13> from 2 on it's purely people.
Oct 25 17:53:39 <f13> lets leave the legalities of 1 asside.
Oct 25 17:53:44 <m_stone> f13: I *really strongly* want to interject that, at step 2, the failure mode is terrible.
Oct 25 17:53:53 <gregdek> We know.
Oct 25 17:54:07 <m_stone> f13: if you are new to the process, as I was --- you have no hope ---
Oct 25 17:54:12 <gregdek> We know.
Oct 25 17:54:14 <f13> There is absolutely no reason that OLPC couldn't have multiple capabile of doing 2, 3, as well as 4
Oct 25 17:54:22 <m_stone> You simply *do not know* the people who know how to make the situation better.
Oct 25 17:54:34 <f13> m_stone: I don't disagree.
Oct 25 17:54:48 <f13> m_stone: which is why we can make the 'people you have to know' be people within your own organization.
Oct 25 17:54:54 <c_scott> f13: but we don't have very many people. I don't actually want to allow K people to do 2, 3, and 4. i want to allow *anyone* (roughly) to do the steps *for themselves*.
Oct 25 17:54:56 <f13> We do that within Red Hat.
Oct 25 17:54:57 <gregdek> m_stone: This is a known pain point. It's the balance between letting anyone contribute, and making sure that only solid contributions are accepted.
Oct 25 17:55:09 <c_scott> otherwise the K people become bottlenecks.
Oct 25 17:55:12 <_bernie> f13, m_stone: I believe anyone can be a package reviewer in Fedora, right?
Oct 25 17:55:19 <f13> c_scott: unfortunately that's just not possible for some of those.
Oct 25 17:55:27 <m_stone> gregdek, f13: So this item is actually a long-term blocker for us.
Oct 25 17:55:33 <f13> _bernie: any current packager can review any other current packager's new package.
Oct 25 17:55:34 <m_stone> At least, I think it is.
Oct 25 17:55:38 <_bernie> f13, m_stone: if we could appoint reviewers "per branch", that would be done!
Oct 25 17:55:45 <gregdek> m_stone: Unless we choose to adopt different review strategies for OLPC content.
Oct 25 17:55:46 <f13> the only gotcha is /brand new/ packagers need a sponsor.
Oct 25 17:55:49 <_bernie> f13: and approve too?
Oct 25 17:55:58 <f13> yes, review and approve for existing packagers.
Oct 25 17:56:09 <f13> let me say that again for clarity
Oct 25 17:56:15 <m_stone> gregdek, f13: look at the kind of growth we're expecting as we start deploying laptops at rates > 100K/month.
Oct 25 17:56:19 <_bernie> gregdek: I like the review checklist in fedora... it improves package's quality a lot
Oct 25 17:56:24 <gregdek> Remember: there is a fundamental difference between *project infrastructure* and *project policy*.
Oct 25 17:56:25 <f13> Any existing Fedora packager can review and approve any other existing Fedora packager's new package.
Oct 25 17:56:29 <f13> period.
Oct 25 17:56:38 <_bernie> See: http://fedoraproject.org/wiki/Packaging/Guidelines
Oct 25 17:56:52 <_bernie> Also See: http://fedoraproject.org/wiki/Packaging/ReviewGuidelines
Oct 25 17:57:01 <f13> The actual review can be a very short process.
Oct 25 17:57:05 <_bernie> And: http://fedoraproject.org/wiki/PackageReviewProcess
Oct 25 17:57:18 <m_stone> gregdek, f13: We simply *can not* have a human bottleneck in the process of becoming a contributor to our software.
Oct 25 17:57:36 <gregdek> Understood.
Oct 25 17:57:39 <m_stone> We can have a human bottleneck about deciding which contributors to bless. or which packages to bless.
Oct 25 17:57:43 <gregdek> That's a matter of OLPC deciding *its* policy.
Oct 25 17:57:50 <gregdek> Which is how it *should* work.
Oct 25 17:58:00 <f13> Of course that requires that the initial packaging is up to spec, and that requires people follow the guidelines when creating packages.
Oct 25 17:58:08 <m_stone> But not in allowing people to get their software packaged and tested on our system.
Oct 25 17:58:19 <gregdek> m_stone: It's a tradeoff, though, yes?
Oct 25 17:58:30 <gregdek> The simpler the packages, the less necessary the restrictions.
Oct 25 17:58:47 <marco> m_stone: well, you cant avoid human bottlenecks in a review process
Oct 25 17:58:47 <f13> the simpler the package the less the restrictions apply
Oct 25 17:59:07 <marco> m_stone: it's not like other olpc code is not being reviewed
Oct 25 17:59:09 <f13> a lot of the restrictions are around more complex packages where you REALLY REALLY want to be sure that you're following the guidelines .
Oct 25 17:59:16 <gregdek> Right.
Oct 25 17:59:24 <gregdek> In a world where RPM dependencies are complex, especially.
Oct 25 17:59:39 <gregdek> WHich is a world that OLPC is wisely choosing to avoid wherever possible by having a sturdy and well-defined base. :)
Oct 25 17:59:45 <f13> We have tools that create up to guidelines spec skeletons which are very easy to fill in.
Oct 25 17:59:52 <m_stone> marco: yes, but this is exactly why package review, IMHO, *must not* be required in order for someone to get their software into a build.
Oct 25 18:00:00 <f13> and then the review is an extremely short procecss.
Oct 25 18:00:10 * djbclark has quit (Remote closed the connection)
Oct 25 18:00:24 <m_stone> f13: here's our problem.
Oct 25 18:00:27 * djbclark (i=dclark@opensysadmin.com) has joined #olpc-meeting
Oct 25 18:00:28 <f13> m_stone: I think we're going to violently disagree here. Packaging /is/ software. There is a /lot/ you can do to ruin good software in a bad package.
Oct 25 18:00:37 <m_stone> package review is a great idea for building quality software.
Oct 25 18:00:42 <m_stone> f13: completely agreed.
Oct 25 18:01:11 <m_stone> f13: and if you still believe that we disagree after, say, five more minutes, then we should table the discussion and move on.
Oct 25 18:01:29 <marco> m_stone: that's a good point
Oct 25 18:01:29 <marco> m_stone: and olpc differ from Fedora there
Oct 25 18:01:31 <marco> m_stone: because develepoment and build are much more integrated
Oct 25 18:02:15 <m_stone> f13: the point that I'm trying to make, though, is that - currently - the four people from OLPC participating in this discussion constitute something like 25% of the people at OLPC who know how to package software for our platform and get it into a build.
Oct 25 18:02:18 <f13> m_stone: Perhaps one hangup is that you feel the package has to be fully approved / imported / etc.. before you can build it in koji and /or integrate it into an OLPC "build".
Oct 25 18:02:18 <marco> f13: the problem is that often we want to get new software quickly into a build
Oct 25 18:02:38 <marco> f13: just so that we can try it out and let other people do so
Oct 25 18:02:52 <marco> f13: and going through a review is overkill there
Oct 25 18:03:01 <gregdek> You know what? I think Fedora's inability to do that is one of its biggest weaknesses.
Oct 25 18:03:07 <m_stone> f13: So if we had as strong and vibrant a community as Fedora does, all of whom were available to help with this problem - required package review might make a lot of sense.
Oct 25 18:03:09 <f13> we dont have that inability
Oct 25 18:03:24 <f13> let me introduce you all to the concept of 'scratch-builds'
Oct 25 18:03:25 <m_stone> f13: explain, pleaseE?
Oct 25 18:03:39 <gregdek> f13: Do we make scratch builds available, though? I'd presumed we didn't.
Oct 25 18:03:50 * gregdek shuts up again. :)
Oct 25 18:03:51 <f13> Koji allows you to throw any source rpm at it, and ask it to build said source rpm against any collection.
Oct 25 18:03:53 <marco> oh can you do scratch builds without approval? that should solve this then
Oct 25 18:03:56 <c_scott> next point on our requirements list: "ability to create short-lived dist tags for experimental or integration streams such as "olpc-xtest", "olpc-sugar", "olpc-trial4", and so on [bernie]"
Oct 25 18:04:02 <f13> gregdek: http://koji.fedoraproject.org/scratch/
Oct 25 18:04:07 <m_stone> c_scott: let him finish, please.
Oct 25 18:04:11 <m_stone> 2 minutes left.
Oct 25 18:04:19 <dgilmore> m_stone: you could have a olpc package taht has all the code you are developing in it, that should be simple to get in for OLPC only and you are free to maintain that as you see fit
Oct 25 18:04:23 <c_scott> i'm just pointing out that we've moved on to the next bullet point
Oct 25 18:04:23 <f13> marco: the only requirement being that you have a valid Fedora account.
Oct 25 18:04:25 <_bernie> c_scott, f13: that's my pet point...
Oct 25 18:04:41 <marco> f13: that solves my concerns then :)
Oct 25 18:04:47 <f13> there is a gotcha with scratch builds, you can't as of yet do a "chain" of scratch builds.
Oct 25 18:04:57 <_bernie> c_scott, f13: it allows creating a sort of distributed development process where we can experiment with things such as X 1.4 without disrupting other people's work.
Oct 25 18:04:58 <f13> you can't scratch A, to rebuild B C and D against it.
Oct 25 18:05:17 <c_scott> f13: which is what the bullet point was requesting.
Oct 25 18:05:26 <f13> riht, that's our next topic
Oct 25 18:05:37 <f13> I was still talking a bit about the previous topic.
Oct 25 18:05:48 <_bernie> marco: sure you can do scratch builds whenever you want... what do you mean?
Oct 25 18:05:48 <m_stone> well, I'm content to segue into the next topic.
Oct 25 18:05:53 <m_stone> our five minutes are up. :)
Oct 25 18:05:58 <f13> The existance of scratch builds should enable you to get your test software built quickly before the review process is started
Oct 25 18:06:00 <c_scott> and again, this is probably at least a partial result of OLPC integrating the packaging and the building more.
Oct 25 18:06:03 <f13> if the software is crap, throw it out.
Oct 25 18:06:07 <f13> if it's good, go through the reivew.
Oct 25 18:06:10 <f13> review.
Oct 25 18:06:11 <f13> make sense?
Oct 25 18:06:25 <m_stone> f13: yes, but it doesn't work for us because we are the upstream authors of packages with dependencies.
Oct 25 18:06:34 <f13> (and yes, I know this is limited to leaf node packages at this point)
Oct 25 18:06:34 <dgilmore> c_scott: we could enable some people to have the ability to create those tags so you can do that work. it could quite easily be someone inside of OLPC
Oct 25 18:06:39 <m_stone> f13: I *still* can't build rainbow in koji because it depends on pyvserver which has not been approved.
Oct 25 18:06:45 <_bernie> f13: the inability to scratch multiple packages was one reason why I couldn't use koji for my X 1.4 work.
Oct 25 18:06:59 <f13> m_stone: right, what you need is the ability to chain-scratch, an I think we can get there in koji.
Oct 25 18:07:07 <marco> _bernie: I thought a package had to be reviewed before you could do a scratch build. Loos like I was wrong
Oct 25 18:07:11 <m_stone> f13: indeed. so the discussion was productive after all. :)
Oct 25 18:07:16 <c_scott> f13: timeframe?
Oct 25 18:07:26 <_bernie> marco: oh, you mean a *new* package? I never tried it out.
Oct 25 18:07:42 <marco> _bernie: yeah
Oct 25 18:07:45 <m_stone> _bernie: we absolutely mean a package that is new to koji.
Oct 25 18:07:55 <m_stone> (regardless of whether or not it is really "new")
Oct 25 18:07:55 <f13> c_scott: that really depends. If you get somebody hired for releng type work, this is exactly the kind of thing they could contribute back to Fedora
Oct 25 18:08:05 <c_scott> heh
Oct 25 18:08:14 <m_stone> f13: casually put. :)
Oct 25 18:08:22 <f13> c_scott: right now /Fedora/ doesn't have a pressing need for this functionality out of koji. Therefor it's not likely it will just show up tomorrow.
Oct 25 18:08:26 <_bernie> marco, m_stone: surely I can build new versions of existing packages... so I could as well rename photoshop9.src.rpm and build that as initscripts.src.rpm.
Oct 25 18:08:45 <c_scott> ok, let's talk briefly about our joyride system, which is what we are using *right now* to work around the things we can't do with koji.
Oct 25 18:08:58 <m_stone> _bernie: if you find a copy of photoshop9.src.rpm that you're legally able to include in Fedora, do let me know... :)
Oct 25 18:09:03 <_bernie> (that's just a stupid example, but it shows that if it's not possible it's just a minor issue)
Oct 25 18:09:28 <c_scott> and let's keep in mind that we have multiple time frames we can work with here -- ie we might decide to do something in the short term, something slightly different in the medium term, and something else entirely in the long term.
Oct 25 18:09:39 <f13> _bernie: marco: yes, you can build brand new, never been seen before source rpms as scratch builds.
Oct 25 18:09:41 <c_scott> adding better scratch builds to koji seems like a medium or long term solution to me.
Oct 25 18:10:04 <m_stone> c_scott: agreed.
Oct 25 18:10:13 <f13> _bernie: marco: you quite literally could build a photoshop package in koji as a scratch build, provided the buildreqs were met.
Oct 25 18:10:18 <c_scott> and short term is "in two weeks we have feature freeze for the final FRS release of our software"
Oct 25 18:10:30 <f13> c_scott: sounds reasonable.
Oct 25 18:10:42 <c_scott> sorry, "feature freeze" is *monday* (sigh)
Oct 25 18:10:55 <_bernie> c_scott: oops
Oct 25 18:11:02 <f13> they sneak up on you
Oct 25 18:11:06 <f13> trust me
Oct 25 18:11:08 * gregdek lols.
Oct 25 18:11:08 <c_scott> ok. can i have a few minutes to briefly describe joyride, and its joys and pains?
Oct 25 18:11:14 <m_stone> c_scott: please do.
Oct 25 18:11:45 <f13> go for it
Oct 25 18:11:58 <f13> (just a note, I'll have to drop off once we hit 7~)
Oct 25 18:12:28 <c_scott> so, we lost a number of contributors who had commit bits to koji quite abruptly a few weeks ago, about the time when we were entering a stable build phase.
Oct 25 18:13:02 <c_scott> we were metering new additions to the stable build for a few weeks and pressure was building to let the developers keep *developing*
Oct 25 18:13:39 <f13> oh man do I know that pressure ;?
Oct 25 18:13:39 <dgilmore> sorry to leave early i have to get home
Oct 25 18:13:47 <dgilmore> will read the rest later
Oct 25 18:13:48 <c_scott> so we forked a deliberately minimal 'joyride' build, which was a collection of simple scripts to build a yum repo from a RPMs dropped in developer's individual 'public_rpms' directories on dev.laptop.org
Oct 25 18:13:48 <gregdek> dgilmore: Thanks dude.
Oct 25 18:14:31 <f13> dgilmore: cheers!
Oct 25 18:14:38 <c_scott> the repo was recreated every hour, checked into git, tagged, and pilgrim was pointed at it (along with the standard koji repo) and a new build was started hourly
Oct 25 18:14:59 <f13> wait, you checked a yum repo, into git?
Oct 25 18:15:04 <c_scott> yes.
Oct 25 18:15:06 <_bernie> dgilmore: bye!
Oct 25 18:15:11 <c_scott> it worked surprisingly well. ;-)
Oct 25 18:15:12 <f13> heh, ooooooh keeeeey.
Oct 25 18:15:25 <c_scott> so we can reconstruct exactly what the repo looked like for any build.
Oct 25 18:15:30 <gregdek> Hm.
Oct 25 18:15:50 <c_scott> and git is smart about sharing files etc
Oct 25 18:16:25 <c_scott> our hourly build process actually was keyed to the git tags on the pilgrim build script, the repo, and the md5sum of the koji repo, which let us tell when there was a 'new' build to be made.
Oct 25 18:16:43 <f13> c_scott: I'm going to want to experiment with this a bit I think, as a side note, for funsies.
Oct 25 18:17:09 <f13> c_scott: that sounds pretty slick
Oct 25 18:17:36 <c_scott> this worked well enough (in the short term) for us to cheaply set up a number of new experimental branches, which we've already mentioned: 'xtest' (for X1.4 integration), 'rainbow' (for m_stone's security integration), and 'meshtest' (which has additional test code which automatically runs on our mesh testbed)
Oct 25 18:17:39 <m_stone> f13: send me an email with your contact info (to michael@laptop.org) and I'll point you at the sources
Oct 25 18:17:50 <f13> m_stone: thanks.
Oct 25 18:18:02 <f13> c_scott: it sounds like a great "testing" environment, a la Debian.
Oct 25 18:18:15 <f13> or unstable as it were.
Oct 25 18:18:19 <c_scott> so you dump packages in (say) ~/public_rpms/xtest, and they go in the xtest build, and when the work you move them to ~/public_rpms/joyride, and everyone else sees them.
Oct 25 18:18:38 <c_scott> the debian experimental/unstable/testing/stable hierarchy was in mind when i wrote the code, yes.
Oct 25 18:18:58 <c_scott> so, the big problems with the system:
Oct 25 18:19:45 <c_scott> a) we quickly lose track of where we're getting things, and what a particular build contains. m_stone has written some integrated changelog stuff which should help.
Oct 25 18:19:53 <m_stone> 1) inability to get a human-readable diff of rpm repos)
Oct 25 18:19:55 <c_scott> the info is in the system, just not human-readable yet.
Oct 25 18:20:00 <m_stone> indeed.
Oct 25 18:20:19 <c_scott> yes, that's a side effect -- we still depend on maintainers to provide good changelogs... hang on a second...
Oct 25 18:20:38 <c_scott> b) it's currently using binary rpms. where are the sources? presumably in someone's git somewhere...
Oct 25 18:20:56 <_bernie> For RH people: this are the xtest builds http://xs-dev.laptop.org/~cscott/olpc/streams/xtest/
Oct 25 18:21:02 <c_scott> so we'd like it to take *srpm* in the dir instead, and automatically build the rpms from that.
Oct 25 18:21:26 <c_scott> then we'd be archiving both the srpm and the rpm in the git tree, etc, etc, and maybe we'd be a little closer to giving mstone his rpm-to-rpm diff.
Oct 25 18:21:28 <_bernie> For RH people: and these are its source rpms: http://www.codewiz.org/~bernie/pub/olpc-bernie/
Oct 25 18:22:00 <_bernie> these are the other experimental branches we have: http://xs-dev.laptop.org/~cscott/olpc/streams/
Oct 25 18:22:02 <f13> c_scott: We might have some tools that would help you
Oct 25 18:22:23 <f13> c_scott: we have something called 'treediff' that will compare two trees of rpms and print out the differences based on rpm changelogs
Oct 25 18:22:26 <c_scott> c) access control? anyone can put stuff in their public_rpms (if they have an account on dev). we're calling that a *feature* at the moment.
Oct 25 18:22:35 <gregdek> Heh.
Oct 25 18:22:40 <f13> *snicker*
Oct 25 18:23:07 <c_scott> f13: i'm mostly serious there -- since we've only got 40-some-odd forked packages, I think that access control is mostly a *social* problem for us
Oct 25 18:23:10 <_bernie> f13: maybe it's not the right time to discuss it, but I seem to remember there were plans to migrate away from CVS and the current scheme of pristine+patches in Fedora, right?
Oct 25 18:23:14 <m_stone> hey - so long as we know where you live, we have enforcement mechanisms aplenty...
Oct 25 18:23:17 <m_stone> :)
Oct 25 18:23:20 <gregdek> Yep.
Oct 25 18:23:21 <gregdek> True dat.
Oct 25 18:23:24 <c_scott> as long as I know what went into a build, I can walk over to the person who put it there and yell at them if appropriate.
Oct 25 18:23:32 <c_scott> opinions on this differ, of course.
Oct 25 18:23:33 <f13> _bernie: urgh, don't remind me.
Oct 25 18:23:35 <gregdek> Only implement things like ACLs when they become clearly needed.
Oct 25 18:24:06 <m_stone> gregdek: I see why you and _sj_ are a natural fit...
Oct 25 18:24:06 <f13> TO be honest, that was our (Fedora people's) first approach at the Fedora package repo. Open by default, close it if you need to
Oct 25 18:24:15 <_bernie> f13: you're going to go through something like the freedesktop.org SCM war, too? :-)
Oct 25 18:24:30 <marco> c_scott: not if we are using binaries, since the changelog cant tell you anything about how these rpms was built
Oct 25 18:24:36 <f13> however to get buyoff to move all Core out there we had to flip it for all the core packages to be closed by default, and open as needed. However new packages added to Fedora get open permissions as of a few months ago.
Oct 25 18:24:46 <c_scott> marco: right. f13: right.
Oct 25 18:24:49 <m_stone> marco: we are going to stop using binaries before FRS.
Oct 25 18:24:51 <m_stone> marco: legal requirement.
Oct 25 18:24:55 <f13> _bernie: ZOD SAYS GIT. YOU MUST OBAY.
Oct 25 18:25:05 <m_stone> marco: (we think, anyway)
Oct 25 18:25:09 <c_scott> ok. so we've got a good short-term solution, but now we need to stop playing around and make a new stable build.
Oct 25 18:25:22 <c_scott> since our unstable is 'joyride', our new stable build will be 'killjoy' =)
Oct 25 18:25:22 * _bernie kneels before Zod
Oct 25 18:25:31 <marco> m_stone: that's monday :)
Oct 25 18:25:58 <m_stone> marco: indeed. :)
Oct 25 18:26:09 <f13> so immediately in my mind, I see 'development' that happens
Oct 25 18:26:11 <f13> ~>
Oct 25 18:26:16 <c_scott> the question is: how do we leverage koji for the stuff we don't fork, while setting up a more sane (and sourceful) build process for our forked packages.
Oct 25 18:26:18 <f13> oops
Oct 25 18:26:45 <f13> in my mind I see "development" happening somewhere like joyride, and as things stabalize in joyride they get brought to koji and integreated into the collection there.
Oct 25 18:27:13 <f13> Koji has some neat features that can help you control a freeze but still allow some development to happen
Oct 25 18:27:14 <gregdek> Yep yep. But by OLPC's rules, perhaps, and not necessarily by Fedora's rules.
Oct 25 18:27:20 <c_scott> f13: and so your challenge question is: why not use the joyride process for everything? ;-)
Oct 25 18:27:55 <f13> c_scott: would you want to throw it at the thousands and thousands of Fedora maintainers?
Oct 25 18:28:04 <c_scott> f13: do we need to?
Oct 25 18:28:15 <_bernie> c_scott, f13: I don't know if we need all this for 46 packages, but bodhi seems like the Fedora solution to this problem
Oct 25 18:28:20 <f13> c_scott: maybe I misunderstood the question.
Oct 25 18:28:41 <_bernie> Please see: https://admin.fedoraproject.org/updates/
Oct 25 18:28:41 <c_scott> more concretely -- one option is to pull SRPMS from koji for unforked packages, but build all our packages ourself from source.
Oct 25 18:29:37 <f13> c_scott: nothing jumps out and says "NOOOO!" immediately
Oct 25 18:29:56 <f13> I can see where there would be some confusion as to where to make a change for OLPC
Oct 25 18:30:09 <c_scott> one good argument for koji is that we will eventually need a more sane user-management process. but koji's ACLs don't (yet) do exactly what we need -- perhaps by the time we need better user-management koji will have that fine-grained control.
Oct 25 18:30:15 <f13> IE has this this being overridden by joyride or do I check in a fix here or....
Oct 25 18:30:32 <f13> c_scott: you'll have to expand upon that a bit
Oct 25 18:30:37 <c_scott> f13: right. marco, for example, has a good relationship for koji for his packages. i'd expect he'll continue to use that.
Oct 25 18:30:50 <f13> currently the ACLs are on the SCM, not in Koji.
Oct 25 18:30:52 <_bernie> c_scott: for packages like sugar that are 100% homebrew, one could also do as you say... we have code in git.
Oct 25 18:31:07 <_bernie> c_scott: for things like hal or X11, I feel safer using branches in Fedora's cvs.
Oct 25 18:31:11 <c_scott> f13: re the second point -- sorry, our glossary didn't separate CVS and Koji. ;-)
Oct 25 18:31:18 <f13> I see.
Oct 25 18:31:23 <_bernie> c_scott: so I can easily diff/merge from F7's upstream
Oct 25 18:31:26 <gregdek> Gentlemen, I need to run. Thanks to everyone who took the time. Is anyone collecting the logs?
Oct 25 18:31:28 <f13> so how much more fine grained do you need in CVS?
Oct 25 18:31:50 <f13> gregdek: if nobody else does, I have a raw log of it
Oct 25 18:31:56 <_bernie> gregdek: me... I'll post them somewhere on our wiki, ok>
Oct 25 18:31:57 <_bernie> ?
Oct 25 18:32:03 <gregdek> _bernie: That's great.
Oct 25 18:32:10 <c_scott> that's another point, _bernie -- we'd like to setup a good process for merging upstream changes from koji into our sources, even when we've forked them (and pushing changes upstream).
Oct 25 18:32:19 <m_stone> _bernie: we'll continue to format them on the "Build system meeting" page.
Oct 25 18:32:20 <gregdek> Bye all. Hope we can talk again soon, when y'alls hair isn't all on fire. ;)
Oct 25 18:32:26 * gregdek is now known as gregdek_gone
Oct 25 18:32:39 <m_stone> gregdek_gone: bye!
Oct 25 18:32:47 <c_scott> the "joyride build from mostly-koji SRPMS" solution brings us back to our first couple of bullet points about managing the upstream/downstream relationship.
Oct 25 18:33:09 <c_scott> i'll pause here for a moment to thank everyone for participating!
Oct 25 18:33:40 <c_scott> i learned a lot about koji's capabilities and architecture, thank you all.
Oct 25 18:33:47 <f13> Honestly, I think we're going to start dropping. It might be best to continue this maybe tomorrow?
Oct 25 18:34:13 <m_stone> that works for me.
Oct 25 18:34:18 <m_stone> shall we negotiate a time by email?
Oct 25 18:34:26 <marco> sounds good
Oct 25 18:34:27 <f13> sounds good
Oct 25 18:34:31 <gregdek_gone> I'll be on PTO tomorrow, but I don't have much more to say on the tech front.
Oct 25 18:34:38 <c_scott> the primary purpose was really to acquiant y'all w/ the limitations we were seeing in how we were doing things
Oct 25 18:34:54 <gregdek_gone> On the Fedora legal/policy front, though, I'd be happy to run interference on any issues you guys come up against.
Oct 25 18:34:56 <c_scott> enumerating good solutions will profit from some time to think.
Oct 25 18:35:18 <_bernie> yeah
Oct 25 18:35:25 <m_stone> can't argue with that.
Oct 25 18:35:26 <c_scott> gregdek_gone: you should probably interact with jg then
Oct 25 18:35:43 <gregdek_gone> Yeah. Certainly seemed like that was his hot-button issue. :)
Oct 25 18:35:43 <c_scott> gregdek_gone: he's got specific issues with the CLA; we should start the long-lead-time process of getting an olpc CLA made.
Oct 25 18:35:53 <_bernie> meanwhile, I'll try to use mbonnet's documentation to setup a demo koji server for my colleagues and managers
Oct 25 18:36:27 <c_scott> yes, i should have said that we're using a parallel strategy to mockup solutions...
Oct 25 18:36:32 <f13> ok. I'm still feeling pretty good that we can streamline the process for ya'll getting content into Koji or forking content within koji, when it comes time for that.
Oct 25 18:36:45 <c_scott> bernie's setting up a local koji, we have the joyride system, mstone worked some on changelogs, etc.
Oct 25 18:36:47 <f13> and that shouldn't get in teh way of being able to do something like joyride for frontline development.
Oct 25 18:36:55 <_bernie> so we can implement something like c_scott's idea of building our forked packages ourselves, but without using a completely different system than our upstream's.
Oct 25 18:37:01 <m_stone> f13: my take-away is that you folks are definitely people who I wish I'd known a few months ago.
Oct 25 18:37:02 <c_scott> f13: i'd like to learn more about the release notes process, too
Oct 25 18:37:12 <gregdek_gone> m_stone: Right on, dude.
Oct 25 18:37:12 * m_stone seconds the release-notes process.
Oct 25 18:37:17 <c_scott> f13: since we're having trouble getting meaningful changelogs out of the repository koji builds for us.
Oct 25 18:37:25 <f13> c_scott: the creation of the content or the process of exploading that content into the tree?
Oct 25 18:37:31 * orospakr has quit ("Ex-Chat")
Oct 25 18:37:31 <gregdek_gone> I'm glad we're getting acquainted now. Better late than never.
Oct 25 18:37:36 * gregdek_gone is really gone now.
Oct 25 18:37:44 <f13> c_scott: hrm, by 'meaningful' you mean something other than the rpm changelog?
Oct 25 18:37:47 <c_scott> f13: starting with, "where does it come from", i guess.
Oct 25 18:38:05 <c_scott> f13: the RPM changelog has only packaging-specific changes; we're also interested in upstream's description of what's changed.
Oct 25 18:38:17 <_bernie> f13: yeah, I never got where the fedora release notes come from...
Oct 25 18:38:27 <c_scott> in their new release, say. it seemed like your 'release notes' addressed that to some degree
Oct 25 18:38:37 <f13> c_scott: yeah, that's hard to capture. Distributions deal in packages, and don't always list what changed upstream .
Oct 25 18:38:47 <f13> Ok, so...
Oct 25 18:38:52 <f13> http://fedoraproject.org/wiki/Docs that's the starting point of our release notes
Oct 25 18:39:06 <f13> it's volunteers that sign up for a "Beat" or subject to write content about
Oct 25 18:39:20 <f13> then the docs group scrape the wiki and translate it into docbookxml
Oct 25 18:39:30 <f13> that all gets shoved into the fedora-release-notes package.
Oct 25 18:40:09 <c_scott> f13: do the 'beats' have any correspondence to specific packages?
Oct 25 18:40:19 <f13> during tree composes, we use rpm2cpio to expload the fedora-release-notes package and grab specific files out of it to put on the filesystem and the CDs.
Oct 25 18:40:22 <f13> c_scott: loose.
Oct 25 18:41:20 <f13> c_scott: we typically care more about subsystems than individual packages. So there is the Desktop beat, etc..
Oct 25 18:41:24 <f13> http://fedoraproject.org/wiki/Docs/Beats for a listing of the Beats
Oct 25 18:42:13 <f13> some beats are package specific, like kernel, but most are more general.
Oct 25 18:42:27 <c_scott> hmm, okay. let's discuss this in a future meeting or via email.
Oct 25 18:42:50 <c_scott> we need to be able to track new schtuff during development at a more fine grain.
Oct 25 18:43:26 <_bernie> f13: ayeeee!! dwmw2 is here!
Oct 25 18:43:26 <f13> nod
Oct 25 18:43:31 <c_scott> m_stone has a proposal for requiring a changelog file for a given RPM to be present before the RPM gets moved from ~/public_rpms to our repo.
Oct 25 18:43:54 <c_scott> so that we can give a better idea to users about what actually is in a specific build (in user-centric terms)
Oct 25 18:43:57 <f13> c_scott: it's not unreasonable to require more verbose rpm changelogs
Oct 25 18:44:03 <f13> since those are available in many places.
Oct 25 18:44:10 <_bernie> (m_stone is summarizing the meeting for david)
Oct 25 18:44:11 <c_scott> well, but we can't require that of unforked packages from koji
Oct 25 18:44:14 <f13> and there are lots of tools designed to grok them.
Oct 25 18:44:16 <_bernie> (david is catching up)
Oct 25 18:44:23 <f13> c_scott: true.
Oct 25 18:44:31 <c_scott> so the question for you is more like, how can we get better information about koji packages?
Oct 25 18:44:36 <f13> c_scott: you can't require, but you can gently hint (:
Oct 25 18:44:57 <f13> c_scott: honestly, there isn't any information beyond the rpm spec and/or the cvs commit log
Oct 25 18:45:00 <_bernie> c_scott: I'll discuss it later or tomorrow with you and m_stone, but maybe we can save the double change logs by cheating
Oct 25 18:45:26 <_bernie> c_scott: I dislike writing 4 different things for each changes I make: git changelog, cvs changelog, rpm changelog and joyride changelog
Oct 25 18:45:32 <c_scott> i hate to bring up debian, but debian requires /usr/share/doc/<pkg>/ChangeLog as well as /usr/share/doc/<pkg>/ChangeLog.debian, where the second is the packaging-specific changelog that is in the SRPM.
Oct 25 18:46:06 <f13> c_scott: that's not too unreasonable, and /many/ of our packages do have that /usr/share/doc/<pkg>/Changelog, should it exist in the upstream.
Oct 25 18:46:06 <c_scott> back to reality, i think what we might want to write is a tool to query koji CVS to find get the bit of the CVS changelog that corresponds to a specific build.
Oct 25 18:46:13 <f13> c_scott: but if it doesn't exist....
Oct 25 18:46:15 <_bernie> c_scott: projects switching to svn and git are *killing* the old ChangeLog files in favor of the changeset descriptions
Oct 25 18:46:32 <f13> c_scott: and funny enough, I do believe that is something we usually check for in packaging review.
Oct 25 18:46:50 <f13> c_scott: the CVS commit isn't going to help you much more.
Oct 25 18:47:04 <c_scott> f13: well, that's good to know. i think pilgrim kills /usr/share/doc/* for space reasons; it may be that we can extract better information from the RPMs we have.
Oct 25 18:47:04 <f13> c_scott: more often than not it's even /less/ verbose than the rpm changelog, or it's exactly the same.
Oct 25 18:47:18 <f13> c_scott: pilgrim kills it to "save space"
Oct 25 18:47:25 <_bernie> f13: I just copy it over from rpm...
Oct 25 18:47:37 <f13> at least thats the reason it previously existed in our livecdtools that was based on pilgrim.
Oct 25 18:47:50 <c_scott> f13: yes, that's still the case.
Oct 25 18:48:07 <c_scott> but that's probably why I forgot that that info was actually in the RPM
Oct 25 18:48:13 <f13> _bernie: we have 'make clog' which will drop a clog file in your pwd based on the last changelog in spec, and then 'cvs commit -F clog' will use it for the commit.
Oct 25 18:48:30 <_bernie> f13: cool! didn't know
Oct 25 18:48:33 <c_scott> ok, i've got to run soon as well...
Oct 25 18:48:48 <_bernie> f13: perhaps we also have make append_an_empty_log_entry_to_spec ?
Oct 25 18:48:57 <c_scott> f13: thanks so much. i'll hack up some code to see what I can extract from /usr/share/doc and the RPM changelog and see if that works "well enough"
Oct 25 18:49:04 <f13> I have 341 Chang* files in /usr/share/doc/*
Oct 25 18:49:07 <f13> on my system.
Oct 25 18:49:23 <m_stone> f13: clearly you have too much software installed, then... :)
Oct 25 18:49:29 <f13> m_stone: clearly.
Oct 25 18:49:32 <c_scott> f13: also, i might need a pointer to mush so that i know the 'right way' to pull the SRPMs from koji.
Oct 25 18:49:39 <f13> I have to drop out too, time to prepare for the 'Sox party.
Oct 25 18:49:42 <m_stone> *mash
Oct 25 18:50:00 <c_scott> m_stone: thanks.
Oct 25 18:50:05 <m_stone> c_scott: and remember that it does so based on FS access; not urls...
Oct 25 18:50:06 <c_scott> f13: yes, that's why I've got to leave as well. ;-)
Oct 25 18:50:21 <f13> m_stone: c_scott http://git.fedoraproject.org/?p=hosted/mash;a=summary
Oct 25 18:50:27 <m_stone> f13: thanks.
Oct 25 18:50:28 <c_scott> m_stone: well, i can clearly screenscrape URLs. but i bet there's a better way. =)
Oct 25 18:50:56 <_bernie> bender:~# ls -la /usr/share/doc/*/Change* | wc -l
Oct 25 18:50:56 <_bernie> 575
Oct 25 18:51:00 <_bernie> f13: I beat you
Oct 25 18:51:04 <f13> there is koji download-build
Oct 25 18:51:33 <f13> if you got a list of new builds you want, download-build will download them for you, should you not have filesystem access
Oct 25 18:51:52 <c_scott> i think i also need the bits which figure out which builds have a given tag
Oct 25 18:52:02 <c_scott> mash might have the code i'm thinking of; i'll take a look at it.
Oct 25 18:52:02 <m_stone> f13: is Koji's XML-RPC api published anywhere?
Oct 25 18:52:08 <f13> m_stone: the source code :/
Oct 25 18:52:15 <m_stone> f13: not a problem.
Oct 25 18:52:17 <f13> m_stone: koji list-api
Oct 25 18:52:23 <f13> is actually /really/ helpful
Oct 25 18:52:29 <m_stone> f13: we're getting very good at inferring APIs from source code... :)
Oct 25 18:52:42 <f13> I prefer to use koji as a python object and let it do all the xmlrpc work. There is a good python api
Oct 25 18:52:51 <c_scott> f13: that's a good trick
Oct 25 18:52:58 <m_stone> indeed.
Oct 25 18:53:11 <c_scott> f13: also don't forget the BC-virginia tech game, if you get tired of waiting for FOX to actually start the freakin' game.
Oct 25 18:53:14 <f13> I"m full of good tricks like this. I really think I could be valuable to you as you're planning these things.
Oct 25 18:53:15 <_bernie> m_stone: you DON'T want to mess with koji's xmlrpc
Oct 25 18:53:29 * _bernie debugged xmlrpc when he installed koji first time
Oct 25 18:53:32 <f13> hehe
Oct 25 18:53:33 <_bernie> :-)
Oct 25 18:53:40 <f13> yeah, use the python API, not the xmlrpc
Oct 25 18:53:42 <c_scott> f13: like _bernie said, it's too bad we didn't have this meeting a few months ago
Oct 25 18:53:54 <_bernie> m_stone: it's err... somewhat obscure...
Oct 25 18:54:06 <f13> ok, /really/ have to go. bye all!
Oct 25 18:54:10 <m_stone> f13: bye!
Oct 25 18:54:14 <_bernie> c_scott: yeah :-)
Oct 25 18:54:14 <c_scott> f13: i'm cscott@laptop.org, just fyi.
Oct 25 18:54:17 <c_scott> f13: bye!
Oct 25 18:54:26 <_bernie> f13: see you tomorrow, thanks for your time
Oct 25 18:55:08 <f13> jkeating@fedoraproject.org works for me, if yo ucouldn't find it already
Oct 25 18:56:11 <_bernie> c_scott: don't trust him... he thinks he's Jesse Keating.