[FreeGeek] QAing build boxes

[Ryan Yeske]

Proposed quality assurance plan for grant and adoptee boxes
-----------------------------------------------------------

The following tests need to be preformed before a box can leave the
shop.  Not everything listed necessarily applies to each particular
box configuration.

- video playback in different formats (acceptable frame rate)
- pdfs, flash, etc are viewable in a browser
- modem can connect (if present)
- ethernet can connect
- wireless can connect (if present)
- parallel port works
- serial ports work
- mouse/keyboard
- usb devices work
- digital camera
- printer test
- umass device (usb harddrive, usb keychain storage)
- memory test
- cdrom test (checksum, throughput)
- dvdrom/burn test (see NOTE)
- video mode testing (resolution, color depth)
- 3d graphics capability (accelerated? acceptable frame rate)
- harddrive (read/write, throughput)
- cpu/general system stress test

NOTE ON CD/DVD BURNING: we want to minimize the number of disks that
     we burn in the shop, obviously we can't burn a disc for every
     machine we build, but anytime we do need a disc for operations,
     it is an opportunity to test a previously untested burner.
     Tested burners can go into built boxes and after passing the read
     test, we can be pretty confident that they will burn discs in
     that system.

I propose installing a directory like the ubuntu Desktop/Examples
folder that contains all the requisite media for these tests, and
perhaps a demo script, or a suite of demo scripts to test the above.
These tests are preformed by the builder, and can also be run by the
end user of the machine, perhaps aiding in diagnosing future problems.
(This will be automatically installed by the network based
installation process, coming soon.)

It should be noted that for tests of things like cameras and printers
(and perhaps modems and wifi cards), even though such devices may not
be going out with the box, the tests should still be preformed with
some known working devices.  We will learn if the host system is
capable of supporting such devices and it ensures that the requisite
software is installed and configured, making it simpler for the end
user in the future to add these components.

The system should pass all of the above tests without requiring the
builder/installer to load the system with any nonfree software/codecs.

All component hardware comprising the finished system should be
recorded with a description of the unit along with the make and model
number.  Also, any components that are responsible for the failure of
any test should likewise be recorded.  We can use this information to
construct (and contribute to existing) databases of known
(un)supported hardware.