Diffusion Processing Pipeline

The diffusion processing pipeline is a collection of scripts and small programs that we use to turn diffusion-weighted MRI data from our collaborators into more sophisticated models, typically streamline "tracts" along which we can compute various metrics. We can visualize and interact with the output tractogram with Brainapp. At the moment, our pipeline is built around DTI --- Diffusion Tensor Imaging --- but this is not the only thing you can do with diffusion-weighted MRI data, and in the future we may want to turn this pipeline into something that has a few more branches.

Right now little of the code used in the processing pipeline is checked into $G, and those programs that are checked in are usually very small and scattered around the source tree. After Brad's new system is in place, he and Jadrian will work on getting all the scripts we use fixed up and checked into a single location. In the text below, the prefix $dbin means /map/gfx0/common0/diffusion/Interface/bin/data, where a lot of scripts currently live.

Note: if you're running the pipeline for the first time on data from a new protocol, you'll probably discover orientation problems or other issues. It's a real drag to wait around for long processes to complete just to discover that the output is bad. Do yourself a favor and do a test run at low resolution!

Pipeline Steps

Step 1: Convert from Original Format

We receive imaging data in a variety of different formats, but we standardize on MRIimage for all of the programs we write. Therefore we have a few converter programs. We assume that every pipeline starts from this step with DWIs from our collaborators. They may give us files that came more or less directly from the scanner, or they may have done some pre-processing like smoothing, distortion correction, or registration. For example, Ron Cohen's group has started using FLIRT to register all the DWIs together.

• $dbin/processNiftiDwis.py --- convert from NIfTI to MRIimage
• $dbin/processMosaics.m (with $dbin/matlabLauncher.py) --- convert from DICOM Mosaic to MRIimage

• $dbin/rotateBvecs.py --- rotate an original list of b vectors to correspond to registration transformations

Step 2: Process DWIs

Here our scripts take MRIimage DWIs as input and create modified MRIimage DWIs as output. We may upsample, smooth, crop, or pad the images. If we have prior knowledge about orientation issues, we may rotate or flip the images in this step before anything else, so that these operations are as inexpensive as possible in a fully automated process.

• $G/bin/mricrop --- crop or pad images to match a specified voxel volume
• $G/bin/mrifilt3 --- resample images to match a specified voxel volume
• $dbin/diffusionResize.py --- by calling the previous two, automatically resample, crop, and pad images to a specified spatial volume in mm, sampled by a specified voxel volume
• $G/bin/mritransp --- permute and/or flip dimensions in a DWI

Step 3: Generate (and Possibly Process) Tensors

We fit tensors to the DWIs using a nonlinear sequential quadratic programming method; the citation for this process is Ahrens MRM '98 . In manual runs, we may discover a tensor orientation issue after they are generated, and correct it at the end of this step.

• $dbin/run_mridfit.py (which calls $G/bin/mridfit) --- fit tensors
• $G/bin/mritensormult --- multiply tensors by a specified transformation matrix

Step 4: Generate Tensor Metrics

We can generate a plethora of scalar metrics from tensors, including FA, trace, and similar things.

• $G/bin/mritensor_v0 --- has a huge menu of options; give it the -- argument to see a list

Step 5: Generate Streamtubes

See also: Tubegen

Most of our studies involve interacting with and computing measures over "streamtubes", integral paths in the field of principle eigenvectors of the diffusion tensors, which we use as a proxy for axon tracts in the brain's white matter. The tubegen program generates a *.sm file that Brainapp uses to visualize the streamtubes.

• $G/bin/tubegen

Step 6: Brainapp Compatibility

Brainapp, which we use to visualize and interact with the streamtubes, assumes a particular directory structure so it can find the data. Unfortunately, in opposition to the "for humans to read, not computers" principle, this directory structure is a little nonintuitive for the naive human user. As such the latest versions of the pipeline generate output in a nested directory hierarchy using simple, plain-English names so that people can navigate it easily. We must map this structure to Brainapp's expected structure to make the program work right.

• No self-contained script exists right now, just a sequence of commands in a Makefile. Check out $G/data/diffusion/brown3t/cohen_hiv_study.2007.02.07/Makefile.patients.

This step also generates a custom Brainapp config file to open the particular file, so that users don't have to manually edit a config file for each new brain they want to view. In the code below, <BRAIN> is the path to the root directory where the pipeline was run for this particular brain, e.g. $G/data/diffusion/brown3t/cohen_hiv_study.2007.02.07/patient003/tubes.2008.07.27.

> cd $G/src/brainapp
> obj/brainapp-gcc3-d -f <BRAIN>/brainapp/settings.cfg -f settings-desktop.cfg

Tips

• Run jobs on a fast machine rather than your workstation or a CS lab machine.
• If you have to do a big batch of datasets, consider running them in parallel on the grid engine.
• Especially when logged in remotely, fork off jobs so that you can log out and keep them running. Remember to use the notify target to get an email sent to you once it's done. In csh:

  make all notify >& make.out &

• DTK is a suite of tools developed by Van Wedeen's group at MGH to accomplish the same thing as our processing pipeline: convert input DWIs into a diffusion model, and then perform tractography. DTK implements not only DTI and DTI tractography, but also q-ball and DSI spherical harmonic ODFs and tractography from those. Therefore DTK can be used for comparison in studies where it is appropriate. Trackvis is the streamline viewing program for DTK, analogous to Brainapp. Both DTK and Trackvis are installed in $G. Note that the DTI fitting algorithm that DTK implements is different than the one we use, and that Trackvis uses its own tractogram file format that is incompatible with Brainapp's.
  • Note that DTK (as of version 0.6.2.2 / 2012.09.20) puts some stupid restrictions on the input format for Q-ball reconstruction:
    • You must use .nii rather than .nii.gz files. This is because of a bug in the DTK GUI: it strips the .nii.gz extension for the internal representation of the input filename, but then adds a ".nii" when generating the command-line calls to its constituent tools. In this case, the process will fail on the second step (odf_recon). You can either run the command-line tools directly yourself, or use gunzip to fix this.
    • You must provide a set of sequentially-numbered 3D volumes (not a single 4-D volume with all of them concatenated). If you don't do this, DTK will give you an error complaining about "multiple high b values", even if your gradient table specifies only one. You can use $G/common/mri/niftisplit to get things the way it likes.

See Also

• Pipeline bugs and feature requests
• In-House Diffusion MRI Software