Build collections

Currently we produce build images; one per build. We should be able to separate highly-independent packages such as library-core from the rest of the build at a deep level.

It is silly to have to spin 12 entire system builds, including pieces of koji and pilgrim, in order to untar a file into a directory and run a user-executable script... just because you need versions of texts and videos and sounds in 12 languages.

I have to talk more to dennis and scott about precedents here. Sj talk

library example

Some practical goals: for library collections that country teams and OLPC are making, I would like to have a small core library available in a few sizes and all supported languages. Different sizes means different collection-trunks; each with an audience and the guiding principles behind how the collection is maintained, which can be formalized somewhat to minimize the frequency of by-hand changes.

Different languages means different best-available content chosen to match the same guiding principles and audience.

Proposed : a database of bundles, collections of bundles, and the priority-sets for each collection to give hints with short turnaround as to the best options for a new collection given available updates and preferences. (so one could autogenerate, say, a good approx. of the best 50M bundle given the known bundle-prefs and stability risk-aversion of the Peruvian team). The database might have tables such as :

bundles : b_id | name | language | audience | maintainers | url
bundle versions :  b_v_id | b_id | version | size | stability | source

collections : c_id | name | languages | audience | maintainers | url
collection versions : c_v_id | c_id | version | size range | priority-lists

collection priority lists : cp_id | name
collection priorities :  p_id | cp_id | priority |
                        stability target | size target

library-core should be a very well-isolated part of the total build -- any links will be library-internal, and almost by definition no part of the library will be allowed to touch other parts of the system.

There will be light dependencies of library-core on library-common and sugar versions. Even this can be improved upon by letting library bundles define what methods they expect will be used to install them, and then defining a set of conversions that can make good guesses at how to turn OCW/SCORM/xol/rpm's into one another.

One key here is not demanding perfection from conversion scripts or collection-generators. All should be tweaked and finalized by people; but if you can automate 95% of this and note when you find a format 'conflict', as with merge conflicts, someone can resolve them [and will be grateful for any hints and warning messages]. Sj talk