User:Mstone/Commentaries/Releases 3: Difference between revisions

From OLPC
Jump to navigation Jump to search
(Remove RASCI table)
 
(37 intermediate revisions by 2 users not shown)
Line 1: Line 1:
We make changes to our software for many reasons; however, we make scheduled (major) releases in order to ''deliver'' significant changes to our downstream partners. Major releases may include interface-breaking changes. They are different from [[Unscheduled software release process|unscheduled (minor) releases]] in that they contain larger and more thoroughly planned changes.
We make changes to our software for many reasons; however, we make scheduled (major) releases in order to ''deliver'' significant changes to our downstream partners. Major releases may include interface-breaking changes. They are different from [[Unscheduled software release process|unscheduled (minor) releases]] in that they contain larger and more thoroughly planned changes. This essay describes what I expect from people, like OLPC, who say they want to make major software releases.


= Framework =
= Framework =
Line 7: Line 7:
# ''planning'': Figuring out what to do.
# ''planning'': Figuring out what to do.
# ''development'': Generating changes which may help to meet the new goals.
# ''development'': Generating changes which may help to meet the new goals.
# ''release'': Integrating the changes in a controlled fashion.
# ''integration'': Integrating the changes in a controlled fashion.
# ''deployment'': Helping downstream partners adapt to the new release.
# ''release'': Helping downstream partners adapt to the new release.


However, this rough breakdown offers little concrete guidance on important issues like:
However, this rough breakdown offers little concrete guidance on important issues like:
Line 30: Line 30:
Here's a word-picture of the problem of making a release drawn from the framework above:
Here's a word-picture of the problem of making a release drawn from the framework above:


efficiency deadlock progress trade desires involved
efficiency deadlock progress trade desires involved triage
starvation feedback objectives strategy resources ship
starvation feedback objectives strategy resources ship
consensus authority informed stale wasted confused supported
consensus authority informed stale wasted confused supported
Line 45: Line 45:
plan believable insane ideal predict guess stall
plan believable insane ideal predict guess stall
critical-path root-cause time budget schedule complex
critical-path root-cause time budget schedule complex

If you aren't using the words in this picture on a regular basis then you should either take a close look at your release's situation or update the picture with whatever words I missed.


= Good Ideas =
= Good Ideas =


== Objectives ==
== Objectives & Resources ==


Write an Objectives page (e.g. [[8.2.0]]) recording a consensus on:
Write an Objectives page (e.g. [[8.2.0]]) recording a consensus on:


* target month.
:* target month.
* development goals and priorities.
:* development goals and priorities.
* lead customers.
:* lead customers.
* feasibility of proposed changes.
:* feasibility of proposed changes.


Tell everyone, prominently, to watch the page so that they are notified when it changes.
Tell everyone, prominently, to watch the page so that they are notified when it changes.

Then publish a list of everyone you're relying on for help. Get consensus on the list.


== Release Manager ==
== Release Manager ==
Line 72: Line 76:


:* ''- Demand evidence and persuasive written arguments. Publicly.''
:* ''- Demand evidence and persuasive written arguments. Publicly.''

== Release Team ==

Software releases involve complex tradeoffs -- and making good decisions about these tradeoffs is hard. To do it well, you should write, publish, and maintain a RASCI table describing the people who are or who need to be:

* ''Responsible'' - the person on the hook -- in this case, the release manager
* ''Approvers'' - the people who are accepting or rejecting the '''R's''' work
* ''Supporters'' - the people who are doing the work for the '''R'''
* ''Consultants'' - the people who the '''R''' relies on for advice
* ''Informed'' - the people whose decision-making depends on the '''R's''' decision-making


== Release Contracts ==
== Release Contracts ==
Line 103: Line 97:
#* You exit ''Ice'' by executing the [[Software ECO process]] and its accompanying [[USR_Checklist|Release Process Checklist]].
#* You exit ''Ice'' by executing the [[Software ECO process]] and its accompanying [[USR_Checklist|Release Process Checklist]].
# Release day. '''Announcement Day'''. Once Release checklist is complete, you're done!
# Release day. '''Announcement Day'''. Once Release checklist is complete, you're done!

== Habits ==

There are several habits of mind and practice which I have found among participants in successful release efforts and against which I judge current ones:

# '''Details matter''', and the definitions need to be negotiated publicly and up front. What do these words and phrases mean: ''architecture'', ''design'', ''X was tested'', ''X is fixed'', ''X will scale'', ''we support'', ...?
# '''Provide evidence.''' You're touching lives, so you had better be able to justify your decisions.
# '''Improve''' the signal-to-noise ratio by shaping conversations to avoid ratholes -- prefer ''"how long should we block on X?"'' to ''"is X a blocker?"''.
# '''Listen''' more and better than you speak... but speak well, and '''teach''' when you do. Moderate conversations to ensure that everyone who needs to be heard, is. '''Practice.'''
# '''Empower the people you trust'''. Leave things and people better than you found them.
# '''Write''', '''speak''', '''illustrate''', and '''test''' your ideas so that you may improve them.
# Above all, get people '''unstuck'''. (Remember that you can't expect people to do things you aren't willing to do yourself.)

== Infrastructure ==

Watch how you and your companions ''are'' collaborating.

Then create affordances to accelerate, clarify, and record those interactions.

Examples from 8.2.0:

:* the [[Trac ticket workflow|next_action]] field,
:* [[Triagebot]],
:* [[Friends in Testing]],
:* [[Test cases 8.2.0]]

Other notes on infrastructure:

# Grant permissions liberally, and take regular backups. Better yet, [http://git.or.cz distribute].
# Be wary of infrastructure that you can't monitor, can't reproduce, or can't integrate with preexisting systems.

== Education ==

Go read some books on [http://www.amazon.com/Mythical-Man-Month-Software-Engineering-Anniversary/dp/0201835959 software development], [http://www.amazon.com/Software-Project-Survival-Guide-Practices/dp/1572316217 release management], [[User:Mstone/Bibliography#Testing|software quality assurance]], [http://www.amazon.com/Strategy-Meridian-B-Liddell-Hart/dp/0452010713 military strategy], [http://www.amazon.com/Essential-Drucker-Druckers-Writings-Management/dp/0066210879 management], [http://en.wikipedia.org/wiki/Critical_Path_Method project management], [http://www.amazon.com/Miracle-Dialogue-Reuel-L-Howe/dp/0866838864 communication dynamics], and [http://www.amazon.com/Beautiful-Evidence-Edward-R-Tufte/dp/0961392177 evidence]. There are lots of other good ones.

Then go talk about what you've learned with other people who've been there before.

Latest revision as of 18:46, 26 August 2012

We make changes to our software for many reasons; however, we make scheduled (major) releases in order to deliver significant changes to our downstream partners. Major releases may include interface-breaking changes. They are different from unscheduled (minor) releases in that they contain larger and more thoroughly planned changes. This essay describes what I expect from people, like OLPC, who say they want to make major software releases.

Framework

Scheduled software releases consist of work on four broad and overlapping topics:

  1. planning: Figuring out what to do.
  2. development: Generating changes which may help to meet the new goals.
  3. integration: Integrating the changes in a controlled fashion.
  4. release: Helping downstream partners adapt to the new release.

However, this rough breakdown offers little concrete guidance on important issues like:

  • what can we reasonably expect to do?
  • how should we divvy up the work?
  • what may go wrong?
  • how do we tell if it's going well or poorly?
  • how can we do it more, better, faster, cheaper, more clearly, etc.?

To answer these questions, I have turned to other tools: control theory, concurrent systems theory, and a limited theory of my companions' psyches.

Control theory is applicable because the goal of a software release is to hit a moving target by making many small changes subject to regular feedback.
Concurrent systems theory is applicable because it provides great vocabulary for describing how the actual work gets done and for its analysis of failure.
Finally, people (psyches) must be considered because we're trying to combine the labor of a fairly specific group of rather quirky and miscommunication-prone individuals rather than of a network of identical processors running identical software.

Observations

Here's a word-picture of the problem of making a release drawn from the framework above:

       efficiency    deadlock    progress    trade    desires  involved    triage
  starvation      feedback     objectives     strategy    resources     ship
consensus     authority     informed     stale     wasted   confused  supported
  competition     race      disagree      risk      affordable   rebuffed
     slip      cut      broken     hacked-up     tested   engaged    burned 
       know    evidence     signoff     stuck     waiting    queue     rest
     consent      responsible    accepted      consulted     contract  sick
 production    merge      freeze      slush      candidate    criteria   vacation
       frequency     magnitude     severity     priority     window   prototype
    blocker     polish     demotivated      process     approved    workflow
 quality     performance     usability    security    correctness    interoperability
   change       integrate       assure      document      release      rebase   use
        delegate       stovepipe      drop        announce      decide      team
   plan       believable     insane       ideal    predict    guess    stall   
      critical-path    root-cause     time     budget     schedule    complex

If you aren't using the words in this picture on a regular basis then you should either take a close look at your release's situation or update the picture with whatever words I missed.

Good Ideas

Objectives & Resources

Write an Objectives page (e.g. 8.2.0) recording a consensus on:

  • target month.
  • development goals and priorities.
  • lead customers.
  • feasibility of proposed changes.

Tell everyone, prominently, to watch the page so that they are notified when it changes.

Then publish a list of everyone you're relying on for help. Get consensus on the list.

Release Manager

You'll save yourself great pain and suffering if everyone agrees, up front, on who is going to bear the responsibility and authority for deciding what's shipping and what's slipping in your release. This person is your release manager.

  • - This person needs to be willing to speak and write in public. Pro-actively.

You need to have a release manager who you can rely on to make the release happen.

  • - Therefore, your release manager needs to be good at noticing people are stuck or who could be more involved and at getting them unstuck or more involved.

You need a release manager who keeps you informed about the tradeoffs that they are making between risks, costs, and opportunities.

  • - Demand evidence and persuasive written arguments. Publicly.

Release Contracts

A release contract is an agreement between the release team and one or more contributors to attempt to integrate some desirable change into a release build.

Release contracts normally describe:

  • the required quality of an acceptable change,
  • a test plan for judging the quality of a proposed change, and
  • who will execute the test plan.

(Typically, violation of a release contract will result in deferral of the proposed change.)

Examples are available. See our Trac conventions for help interpreting the display.

Schedule

I have learned that certain minimum amounts of time must be allocated to integration and testing. I conceptualize this requirement by thinking of the release as passing through several necessary "phases", arranged like so:

  1. >60 days before target date is Steam.
    • Changes can be proposed at will and should be proposed as early as possible.
    • There is great freedom to propose changes because resources have only been loosely allocated toward integrating and testing the proposed changes.
    • The transition to Water occurs when all release contracts are signed.
  2. 60-30 days before target date is Water.
    • This is feature-level change control -- changes requiring reallocation of integration, test, or downstream resources (i.e. requiring a new release contract) are now likely to be deferred.
    • Minor changes can still be added without approval until the transition to Ice. Changes requiring great coordination to deliver like string changes and UI changes will be deferred if possible.
  3. 30 days before target date is Ice.
    • Ice begins when we branch for release so that the release team can produce release candidate builds as needed under package-level change control.
    • The effect of this change control is that fixes -- often for "blockers" or "polish" -- are more selectively merged.
    • You exit Ice by executing the Software ECO process and its accompanying Release Process Checklist.
  4. Release day. Announcement Day. Once Release checklist is complete, you're done!

Habits

There are several habits of mind and practice which I have found among participants in successful release efforts and against which I judge current ones:

  1. Details matter, and the definitions need to be negotiated publicly and up front. What do these words and phrases mean: architecture, design, X was tested, X is fixed, X will scale, we support, ...?
  2. Provide evidence. You're touching lives, so you had better be able to justify your decisions.
  3. Improve the signal-to-noise ratio by shaping conversations to avoid ratholes -- prefer "how long should we block on X?" to "is X a blocker?".
  4. Listen more and better than you speak... but speak well, and teach when you do. Moderate conversations to ensure that everyone who needs to be heard, is. Practice.
  5. Empower the people you trust. Leave things and people better than you found them.
  6. Write, speak, illustrate, and test your ideas so that you may improve them.
  7. Above all, get people unstuck. (Remember that you can't expect people to do things you aren't willing to do yourself.)

Infrastructure

Watch how you and your companions are collaborating.

Then create affordances to accelerate, clarify, and record those interactions.

Examples from 8.2.0:

Other notes on infrastructure:

  1. Grant permissions liberally, and take regular backups. Better yet, distribute.
  2. Be wary of infrastructure that you can't monitor, can't reproduce, or can't integrate with preexisting systems.

Education

Go read some books on software development, release management, software quality assurance, military strategy, management, project management, communication dynamics, and evidence. There are lots of other good ones.

Then go talk about what you've learned with other people who've been there before.