Basic Subtomogram Averaging with I3¶

Maps directory¶

I3 is very selective when it comes to the names of the maps. Shorter names seem to give the least amount of trouble. Therefore it is useful to create symbolic links to the extracted particles in the directory above our I3 project directory with new names to keep the program happy. First, obviously change into the maps directory and the following Bash shell loop does exactly that:

jliu@keemun i3/maps $ i=1; for j in ../../*.mrc
do
    ln -sv ${i} p$(printf "%05d" ${i}).mrc
    i=$((i+1))
done

This creates symbolic links from whatever your subtomograms are named to “p00001.mrc, p00002.mrc, …”

Tilt angles directory¶

Next, the missing wedge for each map is described using our tilt angle files. The most simple and straightforward way to do this is to using the template we copied to our project directory to describe the missing wedge for each map and particle. To do this we again create symbolic links to our template file for each map that we just created in our maps directory. Again, change into the tlt directory and the following loop will accomplish that:

jliu@keemun i3/tlt $ for i in ../maps/*.mrc
do
    ln -sv ../template.tlt $(basename ${i} .mrc).tlt
done

This creates symbolic links “p00001.tlt, p00002.tlt, …” to template.tlt.

Transforms directory¶

The transforms directory can be the most challenging to fill, there are many possible situations based on your particular project:

1. Particle coordinates as a single point per subtomogram center; no orientation
2. Particle coordinates as two points per subtomogram; describes orientation
3. Old I3 transform as a pos file; describes inverse orientation
4. New I3 transform as a trf file from a previous run; describes orientation

In the first case we can create the most basic transform file for each map. Refer to the I3 tutorial PDF to understand what each field of the transform file describes. After changing into the transforms directory the following Bash shell can be used to create these files:

jliu@keemun i3/trf $ i=1; for j in ../../*.mrc
do
    eval $(i3stat -sh -o ${j})
    tx=$(((ox + nx) / 2))
    ty=$(((oy + ny) / 2))
    tz=$(((oz + nz) / 2))
    fmt_i=$(printf "%05d" ${i})
    echo -n "p${fmt_i} ${tx} ${ty} ${tz} " > p${fmt_i}.trf
    echo "0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0" >> p${fmt_i}.trf
    i=$((i+1))
done

In the second case it is easiest to use the programs that take these positions and convert them into Old I3 transform pos files and go forward from the third case. From here we can convert these to New I3 transforms using the i3assist library. To do this we go to the directory with the original maps and position files and start the IPython console using the command ipython from the shell and use the following commands:

import i3assist
import glob
import os.path
for posfile in sorted(glob.glob("*.pos")):
    trffile = os.path.splitext(posfile)[0] + '.trf'
    pl = i3assist.PositionList()
    tl = i3assist.TransformList()
    pl.from_file(posfile)
    pos = pl[0]
    pos_trf = pos.to_trf()
    tl.transforms = [pos_trf]
    tl.to_file(trffile)

Then we need to change directories back to the I3 project transforms directory and run the following Bash script to correctly insert the proper first four fields into the convert transform files and give these files the correct names:

jliu@keemun i3/trf $ i=i; for j in ../../*.mrc
do
    trffile=${j/%mrc/trf}
    fmt_i=$(printf "%05d" ${i})
    eval $(i3stat -sh -o ${j})
    tx=$(((ox + nx) / 2))
    ty=$(((oy + ny) / 2))
    tz=$(((oz + nz) / 2))
    awk -v s="p${fmt_i}" -v tx=${tx} -v ty=${ty} -v tz=${tz} '
        { $1 = s; $2 = tx; $3 = ty; $4 = tz; print }' ${trffile} > p${fmt_i}.trf
    i=$((i+1))
done

We can then safely delete the temporary transform files we created with i3assist.

jliu@keemun i3/trf $ rm ../../*.trf

For the last case we just need to rename the transform files to match the same naming convention as our maps. The loop to do this is simply the one we used for filling the maps directory:

jliu@keemun i3/trf $ i=1; for j in ../../*.trf
do
    ln -sv ${i} p$(printf "%05d" ${i}).trf
    i=$((i+1))
done

Definitions directory¶

The last step in filling in our project directories is the definitions directory, which just contains two files maps and sets. Refer to the I3 tutorial PDF to see the format of these files, but with all of the other directories already setup generating these two files is simple using the following loop:

jliu@keemun i3/defs $ touch maps sets; for i in ../maps/*.mrc
do
    echo "../maps $(basename ${i}) ../tlt/$(basename ${i} .mrc).tlt" >> maps
    echo "$(basename ${i}) $(basename ${i} .mrc)" >> sets
done

And we are done with setup, and can continue to actually processing our project, which is extremely simple.

Troubleshooting¶

This is the most likely command to fail in running I3, due to the fact that this is when the I3 database is first created. The error messages are also not particularly helpful but the following suggestions may help.

When restarting a run that failed delete the cycle-000 directory make whatever corrections necessary and then rerun the command: rm -rf cycle-000 && i3mrainitial.sh.

• i3external errors are often caused by your maps having too long of a

filename or if you have followed this guide that should not be the case, and therefore means you have more maps in project than I3 can handle which can be anywhere from 1,000 to 10,000. Try splitting your data into multiple I3 projects, or refer to the intermediate or expert guides for how to manually add maps to the database.

• i3boximport errors are due to the second to fourth fields of of your

transform file being incorrect. Again if you have followed this guide your particle center coordinates should be correct. However, if you created the transform files yourself, double check that your volumes coordinates and origin correspond correctly with the centers in your transforms. You can do this using the I3 command: i3stat -o <Your subtomogram filename>.

• i3dataset errors are the most difficult to debug. They signal that the

shifts and rotations describing a subtomograms orientation and position are invalid, duplicated elsewhere in the transform file, or incorrectly formated. Again following this guide should prevent these problems, but if you created your own transform files, make sure that each line in the transform file has 16 fields; that lines have the same transformed coordinates, and that the last nine fields are all values between 0 and 1.

Troubleshooting¶

Errors in this stage of processing are uncommon. However if you have any errors, they will almost certainly come from a mistake in the parameter file. Make sure that alignment parameters are sane, and most frequently make sure that the class averages requested were actually calculated.

Rerunning this step to fix specific errors is beyond the scope of a beginner tutorial, and for more information on how to handle these situations efficiently, please refer to the intermediate and expert guides.

Troubleshooting¶

The only error in this stage is if you selected a class for which class averages were not generated. Edit your parameter file making sure that the class selected does exist.

To rerun the command, find the most recent generated directory in the cycle-000 directory and delete it (it should have the name <…>-000-sel):

jliu@keemun i3 $ ls -ltr cycle-000 # Find the most recently created directory
jliu@keemun i3 $ rm -rf cycle-000/<...>-000-sel #replace <...> as appropriate
jliu@keemun i3 $ i3cp.sh 0 # rerun command

Troubleshooting¶

Errors in this stage are also uncommon similar to i3mramsacls and if you encounter trouble here refer to the suggestions for that section.

Troubleshooting¶

You should also not experience any common errors at this stage. Any errors you do experience should point to errors in your parameter files, specifically the filtering, masking, and location of the reference, as well as the masks created for classification.