Software tests

Running Tests

The test driver is installed on all CS department machines. A configuration file is used to launch tests and mail results to subscribers. You can subscribe to a test summary or receive notices when select tests fail. The default configuration file is: $G/common/build/valid/test.platform.cfg

If you want to perform a private test run and suppress mail notifications run:

> /systest/bin/vrl_test  -noemail

To run a set of select tests and specify alternative mail notifications (usually to yourself) you can run using a custom configuration file. Make a private copy of the default configuration file and modify it. The comments in the default configuration file describe the format. Then run using the new configuration:

> /systest/bin/vrl_test  -config  <My work area>/<Custom configuration>.cfg

Here is an example of a small custom configuration file.

#########################  test.platform.fast.cfg  #############################
#
Developer:  you@cs.brown.edu
Summary:    none

nag      none  linux common/build/valid/test.project.py common/include/nag
libnr    none  linux common/build/valid/test.project.py common/gg/libnr
libgg    none  linux common/build/valid/test.project.py common/gg/libgg
libds    none  linux common/build/valid/test.project.py common/gg/libds
libggpp  none  linux common/build/valid/test.project.py common/gg/libggpp
libnum   none  linux common/build/valid/test.project.py common/gg/libnum
libcff2  none  linux common/build/valid/test.project.py common/gg/libcff2
libmri   none  linux common/build/valid/test.project.py common/mri/libmri

#
#######################  end  test.platform.fast.cfg  ##########################

Writing Tests

Create a make test target that runs a test that returns a zero status for success and a non-zero status for an error. The test should be able to run as a batch job (not interactive). Projects with multiple tests can use a target named test.<name> for each individual test and a target alltest to run them all.

Test targets should limit themselves to commands supported on both linux and Windows or use Python scripts. Currently only the "rm -f" and "diff" commands are supported. As always, absolute path names are to be avoided.

Large raw input files such as MRI images should be copied to a directory named $GROOT/data/<project>/test/. Cooked data and test output should go to a test directory inside the project directory. Tests should clear output files from prior runs to avoid false positives. Regression tests that use diff to check output should comapare against a file named: <root name>_base.<suffix>