CCNPreProcess Step-by-step

Foreach -- Creates a looping construct in the flow
- Description/notes
  
  Open up the Foreach and fill the columns. The columns are organized like so:
  - $S = subject name
  - $P = study name
  Fill in a row for each subject / study you wish to preprocess.

SetLogFile -- Changes the name of the log file that is written.
- Description/notes
  Sets the log file such that it will be in the directory:
  subject/study/subject_study.log

UnixCommand -- Runs an application that is not wrapped
- Description/notes
  Here the gunzip command uncommpresses the pfiles in the subject/study/Pfiles directory

PfileCorrect -- Pfile k-space correction
- Input
  pfiles, uncompressed
- Output
  Corrected pfiles (if you choose to write them to disk)
  text output describing number, magnitude, location of k-space outliers
- Description/notes
  Don't detrend, we haven't tested this enough yet.
  Physio correction is not yet implemented.
  Check/correct for outliers greater than 3 or 4 std dev.
  Suggested Outlier replacement type is avg of neighbors.
  You must Skip 1st timepoint if the first image in your pfile is a ref map which it usually always is. Note that you Skip 1st timepoint in your outlier correction even if you are going to keep this timepoint in your statistical analysis.
  Keep the Real-Imaginary data processing option on.
  Select Do not write corrected pfile to disk if you only want to see how many outliers are in the pfile, not correct them.
  If you have saved PfileCorrect output to a log file (e.g. Desk.log) and want to get the summary lines indicating the % outliers and worst outliers for each pfile, run:
```
	cat Desk.log | egrep "Total|Worst|name"
	
```
- Diagnostics
  We typically see about 0.5% outliers in pfiles. Largest outliers are usually 10-12 std dev. If your pfiles have lots more or lots bigger outliers you should probably correct them or maybe ask Andy if something was going wrong with your scan.

SpiralReconLX -- Pfile reconstruction from k-space to image space
- Input
  Pfiles (possibly corrected)
- Output
  Image data, in analyze format.
- Description/notes
  The Output Hdr Dir will hold the first pfile header. You can stick other various logs from other processing steps in this directory too.
  Select the pulse sequence corresponding to your pfiles, or, one that Andy tells you you can use.
  Select Move map option to move the 1st image of each pfile to a separate maps directory and renumber the output analyze images accordingly.
  You should keep the pfile consistency check on unless it is complaining about something you know is OK.
  Do not change any of the expert options unless you really know what you are doing.
- Diagnostics
  You should look at at least 1 of your reconstructed images in your favorite viewer; it is better to look at 1 image from each pfile.
  You can look at 1 slice over all timepoints with the SliceApp cine viewer.
  Look at the mean over time, there should not be any spikes except for the first image in each pfile.
  If you select View results a montage of the first image and a plot of the mean over time will be displayed for you.
  You can run nismaps to look at a mean and variance map.
  You will run nisthresh before AIR, both as a diagnostic and to get the threshold.

Afni3dANALYZEtoAFNI -- Afni program to convert Analyze datasets to AFNI datasets
- Input
  Directory of Analyze files
- Output
  AFNI 3d + time dataset
- Description/notes
  This step is here so Afni3dTqual may be used.

Afni3dTqual -- Afni program to compute a quality index for each sub-brick in a 3d + time dataset
- Input
  One Afni 3d + time dataset
- Output
  A quality index for each sub-brick in the dataset.
- Description/notes
  The quality indexes indicate how different each sub-brick is from the norm. The smaller the value, the closer the sub-brick is to the norm. Please keep in mind that the values returned by Afni3dtqual are to be used as a guide only. The program should not be used to assert that a dataset is good or bad.

UnixCommand -- Runs an application that is not wrapped
- Description/notes
  Here the Afni dataset header that was created by Afni3dANALYZEtoAFNI is deleted.

Nisthresh -- compute brain/background intensity threshold for each image
- Input
  directory of reconstructed analyze images
- Output
  ascii file, 1 column, 1 row per image, with the brain/background intensity threshold for that image
- Description/notes
  Select Write thresholds to file (logs directory is a good location).
  Do not Compress images when finished.
  Go into options and de-select Write thresholded images to disk, because you do not want to threshold the images, you just want to know what the brain/background threshold is.
- Diagnostics
  This program is used both as a sanity check on your images and to get a good threshold for AIR motion correction.
  If the intensities of your images do not follow the expected 2 peak histogram pattern, this program will fail. (You can mess with the options to get it to work if your images are just a bit flat.)
  The threshold should be reasonably consistent over time, with some possible discontinuities at pfile borders.
  People typically choose the smallest threshold value from all timepoints as the AIR threshold. You can get the smallest threshold approximately by just looking at the Y axis of the thresholds plot, and exactly by running the thresholds.txt file thru numeric sort or stats:
```
	sort -n thresholds.txt | head
	stats < logs/thresholds.txt

	
```

UnixCommand -- Runs an application that is not wrapped
- Description/notes
  Here the stats program is run using the output from Nisthresh.

Nismaps -- compute mean and variance maps
- Input
  Directory of analyze images. (Probably your motion corrected air_images directory.)
- Output
  A map (analyze format) of the mean of all images in the analyze directory.
  A map (analyze format) of the variance over time of each voxel timecourse.
- Description/notes
  You can use the mean image for baseline correction in the Niscorrect step.
  Even if you aren't going to run niscorrect, the mean image is handy to have around for making masks.
- Diagnostics
  You can look at the variance map to be sure there are no big areas of unexpected variance. You would expect higher variance around the edge of the brain and in areas of artifact.
  You can note the order of magnitude of the variance by looking at the scale factor (written to output window) and compare that to other subjects in your study.

Nisoverlay -- overlay your first functional on your structural
- Input
  Structural image in analyze format
  First functional image in analyze format
  You must know the brain/background threshold for your functional.
- Output
  Grayscale overlay image indicating how your functional fits, overlays, lines up with your structural image.
- Description/notes
  Enter your functional brain/background threshold value (min) from nisthresh as the threshold value.
  Yes, you must do this step NOW. Not months later when you begin your first GroupTransform. If your functionals are not aligning with your structurals you want to know this before you collect your next subject...
- Diagnostics
  Get those blobs in those eye sockets and keep them there. worse better
  You should load up the overlay image in SliceApp and confirm that the functional is where you think it should be. SliceApp is not currently started automatically from the Desktop because you will need to run SliceApp with extra memory to view the overlay image, which is structural size.
```
 java -ms128m -mx128m SliceApp & 
```

Niscorrect -- outlier correct, detrend, baseline correct images
- Input
  Directory of analyze images. (Probably your motion corrected air_images directory.)
- Output
  Directory of corrected analyze images.
  Possible diagnostic maps from the corrections.
- Description/notes
  You should use the blocking factor if you are detrending, because first and second order trends cannot be fit across sharp jumps in intensity across pfile boundaries.
  The blocking size should be the number of images in each pfile. Subtract 1 if you discarded the first image in each pfile. The blocking size must divide evenly into your total number of images.
  If your pfiles had unequal numbers of images, you can enter more than 1 size, see the niscorrect help page.
  The threshold can be the same threshold you used for AirAlign.
  If you select Save auxiliary files you can look at things like the trends that were removed, map of outlier counts, etc.. But, this data takes up a lot of space.
  Generally Perform baseline correction is a useful thing to do. But, if you have different tasks in different pfiles and are hoping to detect differences across pfiles, the baseline correct will probably remove them.
  The Mean image sets the baseline for each voxel timecourse. You can use a mean image, or any other image. The baseline you select is somewhat arbitrary.
  CCN lab typically removes 1st order (linear) trends, Polynomial order 1. Removing 2nd order trends would also be safe, but, you probably don't want to go higher than that. If you use the Internal outlier correction, set it high, like 5 or 6. This removes outliers before fitting the trend line, so the outliers do not bias the fit. It is not necessary to use this.
  The Afni 3dFourier program is an alternative for linear detrend, it does non-linear frequency based filtering to remove any noise in a given frequency range.
  It is usually helpful and safe to Perform outlier correction, typically 3 or 4 std dev are used. This removes outliers after the trend has been removed.
- Diagnostics
  If you saved auxiliary files, the *_reg_m.img analyze file is a map that attempts to show where the biggest trends were. You can load it as an overlay on one of your functional images in SliceApp to see where the trends were. The *_reg_a.dat file is a pgh txyz format file that has the actual trend line removed at each voxel. It can be loaded into SliceApp as a timeseries viewer, if you have one of your functionals as the image.
  If you didn't save auxiliary files, you could try using Nisdiff to get the absolute value of the difference of all you images before and after niscorrect, and then use Nismaps to take the mean or max of those diffs and look at that as an overlay map in SliceApp. Or, you could run Nismaps to make a variance map and see how much the variance is decreased from the previous variance map you made.

IfisSendMail -- send e-mail notification
- Description/notes
  This is an optional step, you can send yourself an e-mail so you know it is time proceed with the remaining steps.
EndForeach -- Indicates the end of a Foreach loop