[FreeGeek] QAing build boxes

[Alec Smecher]

Hi all,

Speaking from (somewhat outdated) experience, if the list below can't be 
automated, it probably won't be consistently checked. Here's what's 
worked well for me in the past:

1. Automated tool for low-level checking, e.g. boot disk. This tests the 
memory and a few other internals.
2. OS install. This loosely tests the CPU, CD, disk, video adapter, 
mouse/keyboard, etc. and serves as a burn-in.
3. Quick set of additional tests:
 - Network connectivity (wired & wireless)
 - Sound card playback works
 - Modem works (if applicable)
 - USB works

The best thing to do is put together a build sheet on which the config 
and testing can be jotted down quickly; that sheet stays with the 
machine until it leaves the building. I believe David mentioned 
something about one of these; if there's not one available, I can put 
one together.

Anyone have current experience with Linux-based system testing tools, 
particularly something that's packaged with Ubuntu?

Regards,
Alec

Ryan Yeske wrote:
> Proposed quality assurance plan for grant and adoptee boxes
> -----------------------------------------------------------
>
> The following tests need to be preformed before a box can leave the
> shop.  Not everything listed necessarily applies to each particular
> box configuration.
>
> - video playback in different formats (acceptable frame rate)
> - pdfs, flash, etc are viewable in a browser
> - modem can connect (if present)
> - ethernet can connect
> - wireless can connect (if present)
> - parallel port works
> - serial ports work
> - mouse/keyboard
> - usb devices work
> - digital camera
> - printer test
> - umass device (usb harddrive, usb keychain storage)
> - memory test
> - cdrom test (checksum, throughput)
> - dvdrom/burn test (see NOTE)
> - video mode testing (resolution, color depth)
> - 3d graphics capability (accelerated? acceptable frame rate)
> - harddrive (read/write, throughput)
> - cpu/general system stress test
>
> NOTE ON CD/DVD BURNING: we want to minimize the number of disks that
>      we burn in the shop, obviously we can't burn a disc for every
>      machine we build, but anytime we do need a disc for operations,
>      it is an opportunity to test a previously untested burner.
>      Tested burners can go into built boxes and after passing the read
>      test, we can be pretty confident that they will burn discs in
>      that system.
>
> I propose installing a directory like the ubuntu Desktop/Examples
> folder that contains all the requisite media for these tests, and
> perhaps a demo script, or a suite of demo scripts to test the above.
> These tests are preformed by the builder, and can also be run by the
> end user of the machine, perhaps aiding in diagnosing future problems.
> (This will be automatically installed by the network based
> installation process, coming soon.)
>
> It should be noted that for tests of things like cameras and printers
> (and perhaps modems and wifi cards), even though such devices may not
> be going out with the box, the tests should still be preformed with
> some known working devices.  We will learn if the host system is
> capable of supporting such devices and it ensures that the requisite
> software is installed and configured, making it simpler for the end
> user in the future to add these components.
>
> The system should pass all of the above tests without requiring the
> builder/installer to load the system with any nonfree software/codecs.
>
> All component hardware comprising the finished system should be
> recorded with a description of the unit along with the make and model
> number.  Also, any components that are responsible for the failure of
> any test should likewise be recorded.  We can use this information to
> construct (and contribute to existing) databases of known
> (un)supported hardware.
> _______________________________________________
> http://freegeekvancouver.org
> 'Helping the needy get nerdy since the beginning of the 3rd millennium'
> https://lists.resist.ca/cgi-bin/mailman/listinfo/freegeek-van
>