$G/src/jspBin/README.txt
< $G
#gem, Liz Marai 01/24/06
#1. The source code for executables in this directory is located at:
#/map/gfx0/users/gem/closest
#
#2. I have trouble understanding Leor's readme below.
# I know Anwar was supposed to modify the 'config' file and the 'data' file,
# then run Leor's Matlab wrapper (wraps my pipeline). I believe Leor's wrapper
# calls the pipeline repeatedly to process several subjects at once.
# I think it also generates RT style files from the registration pipeline
# output, and presents the stats as a csv spreadsheet.
# Since i can't run the wrapper, i usually resort to running directly the
# pipeline -- instructions for *that* are in trail_contact_areas.txt, located
# in this directory.
#!/usr/bin/less
########################################
# Introduction
Read me for instruction on using the joint statistics pipeline (jsp).
Written by Leor on 29 June 2004.
gem: to install things in bin: gfxinstall3 TesterTempG linux/bin/jspBin/TesterTempG
source of TesterTempG and lit_comp: ~/port/manifolds/src/programs/NurbsDist2Cube
source of driver: /map/gfx0/users/gem/closest/cpt
source of iv to dat: /map/gfx0/users/gem/closest/iv_to_dat/
########################################
# Contents
Introduction
Quick Start
Contents
Overview
Generating a shell script with jspGenScript
Run Config File
Data Config File
Running the shell script
Generating a spreadsheet with jspGenCsv
########################################
# Quick Start
To run the pipeline with the test configuration, type/paste the following commands below into a shell. They should always work.
Generate the script that executes the pipeline:
matlab -nojvm -nosplash -r "addpath('/map/gfx0/tools/linux/bin/jspBin'); jspGenScript('/map/gfx0/data/wrist/jspConfig/jspRunTest.csv','/map/gfx0/data/wrist/jspConfig/jspDataTest.csv'); exit;"
Run the script generated above that executes the pipeline:
/map/gfx0/data/wrist/jspOut/jspScriptTest.sh
View the spreadsheet generated above that contains all computed statistics:
more /map/gfx0/data/wrist/jspOut/jspOutTest.csv
Fin. The computed statistics in the spreadsheet can now be analyzed. Now call the first command with your own configuration files and proceed to generate spreadsheets of computed statistics for your own data.
########################################
# Overview
The joint statistics pipeline is implemented by running a series of programs. The programs generally work by taking input data files, processing them, and generating output data files. The output data files of one program can then be the inputs of the other, and the data is thus succesively processed by a series of programs.
The input data for the entire pipeline consists of:
run config file
data config file
common data files
per-subject-hand
per-bone
open-inventor mesh data files
per-position
transform data files
The config files are .csv's (comma separated values files), a widely supported format that can be imported and exported with facility in matlab and in most spreadsheet programs. The data file locations are specified in the data config file.
The output data of the entire pipeline consists of a .csv with the computed statistics for each directed pair of bones in each hand of each subject hand computed over the contact regions computed at different contact threshold distances, all as specified in the run and data config files. Subjects, the persons whose wrists were scanned, are referred to by id numbers of the form 'E#####'. Hand specifies the right or left wrist, generally L or R.
The pipeline is run by generating a script from config file, running that script. The script copies input data files to a working directory, runs all the programs in series. The last program in the series collects the output from many output files in the working directory into a single .csv.
########################################
# Generating a shell script with jspGenScript
jspGenScript.m is a matlab function file that inputs a data config file and run config file, and from them generates a script that executes the pipeline. The script is output to the location specified in the run config file. Opening this script in a text editor, you can see all the commands that compose the pipeline. This can be useful if errors occurs.
In matlab, with jspGenScript.m included in your matlab path, on the matlab command line
jspGenScript('jspRunConfig.csv', 'jspDataConfig.csv')
will call the function that generates the script, where 'jspRunConfig.csv' and 'jspDataConfig.csv' are the run and data config files with which you would like to run the pipeline.
In a shell, again with jspGenScript.m included in your matlab path, on the shell command line
matlab -nojvm -nosplash -r "jspGenScript('jspRunConfig.csv', 'jspDataConfig.csv')"
will generate the script the same as from within matlab above.
########################################
# Run Config File
The run config file should not need to be changed by regular users. Changes to the run config file are likely required only due to computing system or methodological changes.
The run config consists of rows of key-value pairs. Run config keys:
bone: space separated 3-letter bone abbreviations
fDatCoef: space separated coefficients passed to iv_to_dat
iDistCube: dimensions of cube volume passed to driver
argDist: arguments passed to JointStats
dirBin: bin directory with executables
fileScript: pipeline exec script file that will be generated in dirBin
dirWork: working directory for temporary files
dirOut: directory for output files
fileOutCsv: pipeline output file that will be generated in dirOut
execStats: command to execute main JointStats program
execSshSun: command to execute ssh
execMatlab: command to execute matlab
execTTG: command to execute TesterTempG
execMv: command to move files
execCp: command to copy files
execRm: command to remove files
execMkdir: command to make directories
########################################
# Data Config File
Data config files should be made for all sets of data that you would like to run. Large sets of data can be processed without interaction, but if errors occur it can be useful to run on smaller sets of data when finding and sorting them out. It is recommended to maintain a master data config file in your spreadsheet program of choice, exporting the all or sections of it as .csv data config file inputs for the pipeline.
The data config file consists of rows for each subject-hand that specify five columns, from left to right:
subject id, eg E55555
hand, L or R
transform file pattern with a * for the session number, eg outRT_*_L.txt
scan file pattern with $BONE$ for the 3-letter bone abbreviation, eg E55555L_$GBONE$.iv
template directory, where files not specific to any subject, such as identity transforms, are located
########################################
# Running the shell script
The shell script has been created in the location specified in the run config. Locate it and execute it. Running time varies, expect the pipeline to take minutes per subject-hand-bone-pair. The script outputs the .csv of all joint statistics computed in the location specified in the run config.
NOTE: matlab currently is not able to change the permissions of files when it is running from some machines. when running from one of these machines, matlab will fail to set executable permissions on the script file, and you will have to chmod it manually. the cs technical staff, problem@cs.brown.edu, have been informed and are working on the problem.
########################################
# Generating a spreadsheet with jspGenCsv
The last call in the is to jspGenCsv to generate the .csv output file of all joint statistics computed. You can generate a run script, open it and see this call. The arguments and calling of it are the same as for jspGenScript:
In matlab:
jspGenCsv('jspRunConfig.csv', 'jspDataConfig.csv')
In a shell:
matlab -nojvm -nosplash -r "jspGenCsv('jspRunConfig.csv', 'jspDataConfig.csv')"
Using a master data config file, you can generate config files that contain only newly added data that you need to run, or problem data that you need to rerun on. You can then call jspGenScript to run the pipeline on only the data you need to, and you can then call jspGenCsv with the master data config file to gathers computed statistics from all data together in a single .csv spreadsheet.