TestMtg, 2007-12-02

Attending: Kim, Alex, Scott, Michael, Bernie, Chris

Agenda:

1. Automated test harness(es): getting beyond tinderbox (some discussion on tinderbox is good, too!)
2. GUI testing
3. Activity Testing (Greg's project)
4. Integration, System, Performance, Scaling (everything else)

Automated testing ideas:

• Overarching goal is to have as much automated testing as possible to be able to provide 'reliable' builds.
• Second part of the goal is that testing takes a long time (even tinderbox with old hardware)
• Also, a single test methodology can't catch everything, need mulitple test strategies:
  1. Tinderbox
  2. GUI test tool
  3. Headless tests
     • Link checking the library
     • verifying file systems are appropriate
     • text black list, etc.

Build process view:

1. Collect packages from Cogi; make sure there are SRPMs
2. Pilgrim puts all the pieces together and generates change logs
3. Automated testing
4. Installation: this build only shows up on download.laptop.org
   • Official builds (signed)
   • Test stream, release candidates (signed)
   • Development stream (always unsigned)

Need to get this all on a wiki page for discussion, review and reference for all developers and testers.

One goal of this process is that 'latest' releases (even joyride) should always compile and have passed the automated testing.