Registration of Bones Across Positions - Revision history

Tom Sgouros: /* What we have here is a failure to register */

2014-04-15T14:50:54Z

What we have here is a failure to register

← Older revision		Revision as of 14:50, 15 April 2014
Line 75:		Line 75:
	--configFile config.txt		--configFile config.txt

	Where config.txt is actually the path name of the [registration configuration file] you've created. There are sample configuration files in the $DATA/warmBones directory, and if you use the one that corresponds to the failed job, the registration process will use the data from the bones that succeeded and they don't have to be found again in the next run. This can result in substantial time savings.		Where config.txt is actually the path name of the [[registration configuration file]] you've created. There are sample configuration files in the $DATA/warmBones directory, and if you use the one that corresponds to the failed job, the registration process will use the data from the bones that succeeded and they don't have to be found again in the next run. This can result in substantial time savings.

	Repeat the process above and it should succeed this time.		Repeat the process above and it should succeed this time.

Tom Sgouros: /* What we have here is a failure to register */

2014-04-15T14:50:26Z

What we have here is a failure to register

← Older revision		Revision as of 14:50, 15 April 2014
Line 75:		Line 75:
	--configFile config.txt		--configFile config.txt

	Where config.txt is actually the path name of the configuration file you've created. There are sample configuration files in the $DATA/warmBones directory, and if you use the one that corresponds to the failed job, the registration process will use the data from the bones that succeeded and they don't have to be found again in the next run. This can result in substantial time savings.		Where config.txt is actually the path name of the [registration configuration file] you've created. There are sample configuration files in the $DATA/warmBones directory, and if you use the one that corresponds to the failed job, the registration process will use the data from the bones that succeeded and they don't have to be found again in the next run. This can result in substantial time savings.

	Repeat the process above and it should succeed this time.		Repeat the process above and it should succeed this time.

Tom Sgouros: /* Program outputs */

2014-04-09T23:03:00Z

Program outputs

← Older revision		Revision as of 23:03, 9 April 2014
Line 47:		Line 47:

	The output of the classification step is:		The output of the classification step is:

			* Classified scans in a folder called $DATA/classified_scans.

			* Log files found in $DATA/qsub_output/alpha_qsub.sh.oXXXXXX.Y and $DATA/qsub_output/subset_qsub.sh.oXXXXXX.Y, where XXXXXX is an identifier for the run and Y is the number of some process corresponding to that run. The error output is in files called .eXXXXXX.YY. For successful runs, these files will be empty.

	The output of the registration step is:		The output of the registration step is:

	* ~~You'll find the~~ positions of the bones within a pose in $DATA/auto_registr_results. There are two varieties of file in that directory. The '.txt' files contain the output data, while the '.log' files contain potentially helpful output from the program. If the --delete-log flag is used to invoke the registration step, the logs will be deleted upon successful execution.		* The positions of the bones within a pose, stored in $DATA/auto_registr_results. There are two varieties of file in that directory. The '.txt' files contain the output data, while the '.log' files contain potentially helpful output from the program. If the --delete-log flag is used to invoke the registration step, the logs will be deleted upon successful execution.

	* The '.txt' files above are the output of the various attempts to find the rotations and translations of the various bones. The registration step ends by executing 'collector.py' that consolidates the rotations and translations in the '.txt' files. These outputs are left in the $DATA/collected_results.		* The '.txt' files above are the output of the various attempts to find the rotations and translations of the various bones. The registration step ends by executing 'collector.py' that consolidates the rotations and translations in the '.txt' files. These outputs are left in the $DATA/collected_results.

Tom Sgouros: /* Technical Notes */

2014-04-09T21:24:18Z

Technical Notes

← Older revision		Revision as of 21:24, 9 April 2014
Line 24:		Line 24:
	(For long-running jobs you want to leave and come back to, the [[screen program]] is your friend.)		(For long-running jobs you want to leave and come back to, the [[screen program]] is your friend.)

	The result of this step is a set of scripts in $DATA/qsub that will be used for the classification and registration steps.		The result of this step is a set of scripts in $DATA/qsub that will be used for the classification and registration steps. The classify.py script executes the classification step, and unsurprisingly, register.py does the registration. You'll also see a collector.py, which takes the output from the registration step and condenses it to a set of simple transposition matrices. This script is run automatically at the end of the registration step, for the poses where registration succeeded.

	===Program ~~input~~===		===Program inputs===

	The input to the classification stage is:		The input to the classification stage is:
Line 42:		Line 42:
	Configuration files are generated each time the registration step is run, and put into the $DATA/warmBones directory. You can use these files as raw material for creating a configuration file for your experiment.		Configuration files are generated each time the registration step is run, and put into the $DATA/warmBones directory. You can use these files as raw material for creating a configuration file for your experiment.

	===Program ~~output~~===		===Program outputs===

	The output of the two steps consists of files within the input data directory. For example, if the input data is in a directory called E50072_scan, the output of both steps will be files in subdirectories of that directory. In the following, we'll refer to the input data directory as $DATA.		The output of the two steps consists of files within the input data directory. For example, if the input data is in a directory called E50072_scan, the output of both steps will be files in subdirectories of that directory. In the following, we'll refer to the input data directory as $DATA.
Line 50:		Line 50:
	The output of the registration step is:		The output of the registration step is:

	* You'll find the positions of the bones within a pose in $DATA/auto_registr_results . There are two varieties of file in that directory. The '.txt' files contain the output data, while the '.log' files contain potentially helpful output from the program. If the --delete-log flag is used to invoke the registration step, the logs will be deleted upon successful execution.		* You'll find the positions of the bones within a pose in $DATA/auto_registr_results. There are two varieties of file in that directory. The '.txt' files contain the output data, while the '.log' files contain potentially helpful output from the program. If the --delete-log flag is used to invoke the registration step, the logs will be deleted upon successful execution.

			* The '.txt' files above are the output of the various attempts to find the rotations and translations of the various bones. The registration step ends by executing 'collector.py' that consolidates the rotations and translations in the '.txt' files. These outputs are left in the $DATA/collected_results.

	* During the optimization step, the qsub program is used to run instances of the registration program on many different host machines. The standard output of those runs can be found in files called $DATA/qsub_output/register_qsub.sh.oXXXXXX.Y, where XXXXXX is an identifier for the specific run and Y is the job number. The error output is in files with the suffix eXXXXXX.Y.		* During the optimization step, the qsub program is used to run instances of the registration program on many different host machines. The standard output of those runs can be found in files called $DATA/qsub_output/register_qsub.sh.oXXXXXX.Y, where XXXXXX is an identifier for the specific run and Y is the job number. The error output is in files with the suffix eXXXXXX.Y.

Tom Sgouros: /* Technical Notes */

2014-04-08T15:32:17Z

Technical Notes

← Older revision		Revision as of 15:32, 8 April 2014
Line 15:		Line 15:
	To run the classification and registration process, follow these steps:		To run the classification and registration process, follow these steps:

	#~~check~~ whether a "crop_values.txt" file exists-- it helps reduce the number of voxels processed		#Check whether a "crop_values.txt" file exists -- it helps reduce the number of voxels processed.
	#cd to the top-level of your subject's data		#cd to the top-level of your subject's data.
	#ssh to a CS-LAB machine, e.g., "ssh cslab9g"		#ssh to a CS-LAB machine, e.g., "ssh cslab9g".
	#~~run~~ install_linux/bin64/batchGen.py and answer the questions		#Run install_linux/bin64/batchGen.py and answer the questions. (The first time you process the data, accept all the defaults.)
	##The first time you process the data, accept all the defaults		#Type "./classify.py" wait a few hours.
	#~~type~~ "./classify.py" wait a few hours		#Type "./register.py" wait an hour or so.
	#~~type~~ "./register.py" wait an hour or so

	(For long-running jobs you want to leave and come back to, the [[screen program]] is your friend.)		(For long-running jobs you want to leave and come back to, the [[screen program]] is your friend.)

			The result of this step is a set of scripts in $DATA/qsub that will be used for the classification and registration steps.

	===Program input===		===Program input===

	~~The wrist registration program runs in a~~

	The input to the classification stage is:		The input to the classification stage is:

Tom Sgouros: /* Program input */

2014-04-08T15:13:22Z

Program input

← Older revision		Revision as of 15:13, 8 April 2014
Line 27:		Line 27:
	===Program input===		===Program input===

			The wrist registration program runs in a

			The input to the classification stage is:

			* CT scans, stored in a directory called $DATA/original_scans
			* A crop_values.txt file.

			The input to the registration step is:

			* The output from the classification stage, stored in $DATA/classified_scans.
			* The digitized (segmented) bone shapes, stored in $DATA/original_stacks.

			Optionally, the input to this stage can include a [[registration configuration file]], which is used to set starting points for a registration match search, or to limit the range of such searches, or to eliminate the need for a search altogether, on a per-bone basis.

			Configuration files are generated each time the registration step is run, and put into the $DATA/warmBones directory. You can use these files as raw material for creating a configuration file for your experiment.

	===Program output===		===Program output===

Tom Sgouros: /* Program output */

2014-04-07T21:22:11Z

Program output

← Older revision		Revision as of 21:22, 7 April 2014
Line 39:		Line 39:

	* During the optimization step, the qsub program is used to run instances of the registration program on many different host machines. The standard output of those runs can be found in files called $DATA/qsub_output/register_qsub.sh.oXXXXXX.Y, where XXXXXX is an identifier for the specific run and Y is the job number. The error output is in files with the suffix eXXXXXX.Y.		* During the optimization step, the qsub program is used to run instances of the registration program on many different host machines. The standard output of those runs can be found in files called $DATA/qsub_output/register_qsub.sh.oXXXXXX.Y, where XXXXXX is an identifier for the specific run and Y is the job number. The error output is in files with the suffix eXXXXXX.Y.

			* You'll find configuration files that record the bone positions found in the $DATA/warmBones directory. If you need to re-run your job, these files can save some time, since they use the optimized positions found in the registration run. If any pose failed, the configuration file for that pose will include the correct positions of the bones found up to the failure point.

	===What we have here is a failure to register===		===What we have here is a failure to register===

Tom Sgouros: /* What we have here is a failure to register */

2014-04-07T21:20:00Z

What we have here is a failure to register

← Older revision		Revision as of 21:20, 7 April 2014
Line 43:		Line 43:

	The registration step often fails on one or more of the poses in a subject's data. You'll know the run has failed from the output to your terminal. It will say something like		The registration step often fails on one or more of the poses in a subject's data. You'll know the run has failed from the output to your terminal. It will say something like


	ERROR: The WristRegistration runs failed with status = 1		ERROR: The WristRegistration runs failed with status = 1
Line 50:		Line 49:
	Probably only one or two of the poses failed; you can identify them from the text immediately before this. (An exit code of zero means success.) Note that the job numbers you see on the program output are not the same as the numbers indicating a pose. For example, a normal set of poses might be 2,3,4,5,6,7,8,9,10,11,12 and 15. These will be executed as job numbers 1,2,3,4,5,6,7,8,9,10,11, and 12.		Probably only one or two of the poses failed; you can identify them from the text immediately before this. (An exit code of zero means success.) Note that the job numbers you see on the program output are not the same as the numbers indicating a pose. For example, a normal set of poses might be 2,3,4,5,6,7,8,9,10,11,12 and 15. These will be executed as job numbers 1,2,3,4,5,6,7,8,9,10,11, and 12.

	Once you identify the offending pose, you need to use the Wrist Viewer program ([[WristViz]]) to find a good initial position for the bone alignments that failed. When you ~~re-run~~ the ~~registration step~~		Once you identify the offending pose, you need to use the Wrist Viewer program ([[WristViz]]) to find a good initial position for the bone alignments that failed. Create or copy a configuration file and identify it in the corresponding $DATA/qsub/registerXX_qsub.sh file, where XX is the job number of the run you need to fix. When you open that file, you'll see a messy long line that runs the WristRegistration program. Add this to the end of that line:

			--configFile config.txt

	~~This manually-found initial position information goes into~~ files ~~called "config_01L.ini"~~, ~~for example~~, and ~~should~~ be ~~saved into~~ the ~~subject's root data directory~~.		Where config.txt is actually the path name of the configuration file you've created. There are sample configuration files in the $DATA/warmBones directory, and if you use the one that corresponds to the failed job, the registration process will use the data from the bones that succeeded and they don't have to be found again in the next run. This can result in substantial time savings.

	Repeat the process above and it should succeed this time.		Repeat the process above and it should succeed this time.

Tom Sgouros: /* What we have here is a failure to register */

2014-04-07T20:55:38Z

What we have here is a failure to register

← Older revision		Revision as of 20:55, 7 April 2014
Line 45:		Line 45:


	ERROR: The WristRegistration runs failed with status = 1		ERROR: The WristRegistration runs failed with status = 1
	They need to be rerun.		They need to be rerun.

	Probably only one or two of the poses failed; you can identify them from the text immediately before this. (An exit code of zero means success.) Note that the job numbers you see on the program output are not the same as the numbers indicating a pose. For example, a normal set of poses might be 2,3,4,5,6,7,8,9,10,11,12 and 15. These will be executed as job numbers 1,2,3,4,5,6,7,8,9,10,11, and 12.		Probably only one or two of the poses failed; you can identify them from the text immediately before this. (An exit code of zero means success.) Note that the job numbers you see on the program output are not the same as the numbers indicating a pose. For example, a normal set of poses might be 2,3,4,5,6,7,8,9,10,11,12 and 15. These will be executed as job numbers 1,2,3,4,5,6,7,8,9,10,11, and 12.

Tom Sgouros at 20:55, 7 April 2014

2014-04-07T20:55:14Z

← Older revision		Revision as of 20:55, 7 April 2014
Line 44:		Line 44:
	The registration step often fails on one or more of the poses in a subject's data. You'll know the run has failed from the output to your terminal. It will say something like		The registration step often fails on one or more of the poses in a subject's data. You'll know the run has failed from the output to your terminal. It will say something like

	~~[[code]]~~
	ERROR: The WristRegistration runs failed with status = 1		ERROR: The WristRegistration runs failed with status = 1
	They need to be rerun.		They need to be rerun.
	~~[[code]]~~

	Probably only one or two of the poses failed; you can identify them from the text immediately before this. (An exit code of zero means success.) Note that the job numbers you see on the program output are not the same as the numbers indicating a pose. For example, a normal set of poses might be 2,3,4,5,6,7,8,9,10,11,12 and 15. These will be executed as job numbers 1,2,3,4,5,6,7,8,9,10,11, and 12.		Probably only one or two of the poses failed; you can identify them from the text immediately before this. (An exit code of zero means success.) Note that the job numbers you see on the program output are not the same as the numbers indicating a pose. For example, a normal set of poses might be 2,3,4,5,6,7,8,9,10,11,12 and 15. These will be executed as job numbers 1,2,3,4,5,6,7,8,9,10,11, and 12.