Mark Dow

Space Software

Volume segmentation notes

    Practice and procedures for isolation and rendering of structures and regions in volume (3-D) data using Space Software.

    Currently a single example of isolating several regions from a particular volumetric data set, from a T2-weighted MRI sequence, is roughly documented.

T2 slice through cochlea skin surface after masking of noise

Basic ideas


Quick and dirty rendering of a region of interest


Volume bounding a region of interest (ROI)
Create ROI mask

Fill in the selection mask
Fix up masked ROI

ROI mask examples and particulars

Segmenting head from exterior noise
Skin surface renderings

Apply ROI mask

Render ROI

Data files

Programs and code

[To Do: Reference to edge detection material, animation clips, virion segmentation, hippocampi ...]


    Region of interest definition based on complex volume data is difficult, requiring a priori knowledge of structure and visual-manual intervention.This is an image processing problem called segmentation. A variety of techniques developed for 2-D images find natural extensions in 3-D. 

    Segmentation estimates of  location of surfaces, boundaries in a 3-D space, that usually correspond to chemical-density interfaces in the material that was volumetrically sampled. Potential locations of surface points can be automatically generated with gradient operators -- surfaces are near inflection points of gradient magnitude distribution. While Space Software does generate and use this type of gradient information, it is not explicitly used in these examples. The rendering of surfaces does implicitly use gradient information to represent surface.

    3-D manual or semi-automatic techniques require a suitable user interface that Space Software provides. How is Space Software used to define and visualize regions of interest? How could the software be improved and what auxilliary tools would be most useful?

To Do: Why Space Software? What other tools are available, and what are their strengths and weaknesses?

Basic ideas

    Here we are concerned with selecting all or most volumetric elements that contain surface points of a structure, and excluding surface points that are not part of the structure. The method is to create and apply a mask to the volume data, including only the desired surface points within a binary mask.

    A mask is layered on, or on a portion of, the raw data volume. Semi-automatic and manual editing tools are used to populate the mask. The mask is applied, and the remaining surface elements are rendered, or represented and quantified.

Inner ear MRI example

    This segmentation of inner ear structures and nearby nerves and arteries from an MRI T2-weighted image was done primarily for the purpose of rendering, not volumetric assessment or other measurement.

Image acquisition

MRI volume data information
Subject: Mark Dow, T2 weighted brain
Date of scan: January  5, 2010
MRI system:
Siemens Allegra (3T head only), LCNI, University of Oregon
phased array head coil (not the standard quadrature head coil)
Sequence: t2_tse3d_vfl_trans
Investigator: Mark Dow

See data files for download.

Convert scanner's DICOM data to FSL's NifTi-1 (.nii) format, 16-bit:

MRIConvert for data set selection and conversion


single file (.nii) format

FSL and NifTi-1.1 (accessed 2/8/10)

"FSL has supported input and output of NIfTI-1 files since the release of version 3.3 in April 2006. This is the second release of FSL that supports NIfTI-1, and more features are implemented that in the previous release, although not every feature is currently fully implemented.

In order to maintain backward compatibility the current version of FSL (3.3) also supports Analyze input and output. As NIfTI-1 allows voxel coordinates to be mapped onto real-world (mm) coordinates with a transformation (qform or sform) that either preserves left-right orientation of the coordinate system (positive determinant) or reverses left-right orientation (negative determinant) it was necessary to assume that all Analyze files and any NIfTI file which did not specify either qform or sform, had a reversed left-right orientation (also called "radiological convention" in the FSL help pages). This was done in order to be consistent with the avg152T1 standard image."

Create an adjusted Space volume (.vol) format, 8-bit

To Do: Repost, and link to, file specification.

Load NifTi volume into Space Software.
Add volume info comment.
Volume | Volume Information
To Do: Fix the Space editing of Volume | Volume Information | Description  and Title comment text. Apparently the focus is reinitialized, sending the cursor back to the corner at every key stroke.

Adjust brightness/contrast/gamma

The point of this step is to maintain important information about surfaces and textures in the conversion to an 8-bit format.

It is handy to see a histogram of voxel luminance (values), native and transformed, while making these adjustments.

Open the Colors | Transform and Histogram dialog box.
Colors | Gamma
Note that adjusting gamma affects the gradient distribution as a function of luminance, an most surfaces are defined with respect to relative gradient magnitudes.
For example, a positive gamma applied to the volume will increase the number and strength of surface features at low luminances compared with those at high luminance. Another way of saying this is "contrast is increased for low luminance regions, and decreased at high luminance regions".
[To Do: It would be nice to know a gradient magnitude spectrum as a function of:
At inflection points only? ]
Colors | Brightness/Contrast

Select the full range of values that are useful for segmentation.

Apply transform

Colors | Apply Transform to Values

Compress to 8-bit values

Colors | Color Depth | 8-bit Grayscale

Rescale (min/max),?

It is easiest to work with 8-bit values that are numbered as 8-bit integers. Any scaling is equivalent, but more confusing.
Colors | Transform and Histogram | min. = 0 and max. = 255

Save, and save a copy

Most of the operations (creation of masks and masked volumes) use this 8-bit volume. When performing merge operations, it is easy to make the mistake of overwriting this file with a modified version. The copy is used to easily recover from this error.
File | Save As
Append "_8-bit" to filestem, save in .vol.gz format.
Append "_copy" to filestem, save in .vol.gz format.

Choose one or more regions of interest for segmentation.

No good hints here, application dependent. Don't bite off more than you can chew, but theres a cost to endlessly nibbling tiny pieces.

It is better to segment largest (superset) features first, as it can save some work at shared boundaries.

But only if segmentation of the largest features are useful.

Splicing multiple pieces together can be done. View adjacent masks to insure continuity between volumes bounding parts of the ROI (see below).

Quick and dirty rendering of a region of interest (ROI)

    It is handy to get an idea of the structure and data quality of a region and structures that are to be segmented. It is difficult to get a feel for 3-D structure from slices at arbitrary angles, and rendering a region in various ways helps getting a hold on of the lay of the land. This can be done on the raw 16-bit volume, but it's a bit more efficient and just as good to use an adjusted 8-bit volume.

Load volume into Space Software.

Load either an .nii or .vol format file with File | Open Volume

Space does not read DICOM (.dcm) files.

Center volume origin on screen, if not already centered, with Volume | Volume Info | Center

This operation changes the Volume | Volume Info | Origin. Don't resave this volume, or save as a unique copy, if you want to preserve the original origin in the header information.

Slice navigation -- looking at slices of the volume data at any slice angle and position

left click drag for rotations

right click drag for translation in depth

To Do: ...

Set brightness/contrast

b-key mouse (Colors | Brightness/Contrast), or Colors | Transform and Histogram

Crop and resample if desired. Cropping can make navigation while rendering easier if there is a lot of distracting surroundings. Resampling to a smaller voxel size (oversampling) allows for less aliasing in the rendering, and can correct for anisotropic voxel size.

Select crop rectangle

To Do: ...


To Do: ...

Repeat cropping in third dimension

To Do: ...

Window the region of interest (if not cropped appropriately)

w-key (View | Window) and select rectangle 


r-key, and/or Rendering | Rendering parameters dialog box controls

Example quick and dirty renderings

    These examples were done without cropping, and only oversampling to achieve an isotropic voxel size. Less obstructed and chunky (aliased) renderings can be achieved selective cropping and oversampling (To Do: Example.):

See data files, [To Do...]

Volume | Resize (resample) | Isotropic x = .5, y = .5, z = .5 mm  (from raw voxel size of x = .5, y = .5, z = 1.0 mm

T2-weighted MRI slices through left inner ear structures.

inner ear

vestibular system


semi-circular canal
Slices through left inner ear structures from raw data

T2 slice through cochlea
Slice through cochlea,
with brain's temporal lobe
directly above.
Renderings of right inner ear
structure from raw data, at a
variety of thresholds.

At the top of each image
brain (temporal lobe) partially
obstructs the view. the primary
background is (bright)
cerebrospinal  fluid (CSF) 
surrounding the cerrebelum. In 
the top image, the arc of the
facial nerve passes directly  in 
front of the inner ear structures.

Stereo pairs, cross-view.
Renderings of right inner ear structures from raw data, stereo pair Renderings of right inner ear structures from Gray's Anatomy
Drawing of inner ear from Gray's anatomy

Renderings of right inner ear structures from raw data
Rendering from about the same viewpoint as the Gray's anatomy drawing.

Rendering of skin surface, noise, and spike artifacts Rendering of skin surface and bottom of volume

Volume bounding a region of interest (vbROI)

Crop to vbROI
Volume | Select Crop Rectangle
Volume | Crop
Do not change the origin. It will be off center, but will stay registered with other ROIs.
Use windowing (View | Window) to center the image on the screen.
View | Volume Cues are not available in the windowed state, unfortunately.
To Do: Add volume cues unique to the windowed state to Space.
To Do:
Cropping does not work properly in the windowed state; the crop rectangle is displayed at the wrong location.
Volume | Resize (resample)
Set brightness and contrast defaults
b-key mouse, then Colors | Transform and Histogram | Apply as Default.
These get changed frequently during mask creation, so it's nice to easily return to a default view (Colors | Transform and Histogram | Default All, or Colors | Use Default Color Transform.
Rename and Save

Create ROI mask

    All the manual nitty-gritty work is done here. Save often. Invariably this mask will require tweaking.

Create a blank overlay to hold the mask

Overlays (or File) | New Blank Overlay
Overlap the current volume, 8-bit grayscale
Colors | Transform and Histogram
Transparency to about 70%
Coloring (so mask voxels are easily distinguished):
Red channels White Point to 1255 (just prepend the 1).
This has the effect of  partially removing red leaving the mask voxels cyan, so they can be distinguished easily from underlying overlays.
Apply as Default
This is often tweaked later, and its not convenient to reset these values every time.
File | Save As and rename
    Append "[name]_mask[#]" to filestem
Fill in the selection mask

Use Edit, Edit | Edit Toolbar tools:

Edit | Select Edit Color
black or white

Fill 3-D
 Tool properties
Fill (2-D)
 Tool properties
 Tool properties
Gaussian blur for expansion
Volume | Make Filtered Overlay | Gaussian

First rough cut

To Do: For regions with extensive continuous contrast boundaries:
 e.g. some inner ear structures

To Do: For large regions poor contrast boundaries:
e.g. brainstem

Work with a copy of the vbROI with a shortened filestem. Easier to recover from incorrect saving mistakes.
When editing plane-by-plane:
Use "<" and ">" keys to navigate to the next View | Slice Positioning | Next or Previous Slice
Use an orthogonal orientation, o-key or View | Coronal, Sagital, or Axial
A lot of switching between mask and vbROI is required. The mask volume is being edited, but the contrast/brightness of the vbROI is frequently changed.
Use Edit | Overlay Viewer for selection of current volume, or get used to n-key for cycling between volumes.
Also use the  h-key (or the "eye" button on the Overlay viewer) to toggle visibility of the mask (Overlays | Hide this Overlay).
The Colors | Transform and Histogram or Edit | Select Edit color dialog boxes serve as good visual indicators of the current volume. These dialog boxes are also used frequently for other tasks, e.g. toggling back to default viewing parameters.
First cut 3-D fill
The fill tools operate on the displayed luminance of the displayed over/underlays.
Set the brightness/contrast (b-key mouse position) of the volume bounding this ROI, below the mask volume in the overlays list (Edit | Overlay Viewer).
Remove excessive bleeding by redoing after resetting the Max. voxel extent in the Edit | Tool properties (with Fill 3-D active) to a smaller value.
If this isn't noticed before a lot of editing is done, it is difficult to recover gracefully.
Try using Overlays | Mask with this Overlay while editing the mask edges.

Fix up flaws in mask

Morphological smoothing
    Apply 3-D erosion and dilation operators to the mask to remove unsupported voxels and fill in small gaps. This is currently implemented as a MATLAB program, not as a part of Space Software. This will reduce the need for manual fine tuning and make the regions that require manual intervention more obvious.

Save the mask in Space (.vol) format, without g-zipped compression (.gz).

Apply operators:

MATLAB command:
>> remove_if_unsupported( '000001_5_4_t2_tse3d_8b_vb_brainstem_mask1', 3, 4 );
To Do: Rename, document, post.
Open "_clean" volume in space
reset origin to match original mask

Save as the most current mask
Gaussian smoothing

If smoothing is used, return to binary mask -- applying a feathered mask might work for bright to dark boundaries, but has ill effects for dark to bright boundaries,
Fine tuning:

Use 5x5 circular Pen tool on one feature or portion of a long edge, while stepping through planes.
To Do: Get a cheap pen tablet and make it work with Space editing tools.
To Do: How to handle saddle points?
To Do: Isolating ROIs with a planar boundary.
ROI mask examples and particulars

Segmenting head from exterior noise

    This T2-weighted MRI volume has noise and artifacts that are peculiar to the sequence, but not at all uncommon for MRI in general. Often it is preferable to minimize the effects of noise on rendering or surface determination.

Whole volume renderings at with and without various noise artifacts:
Rendering of skin surface, noise, and spike artifacts
Top: Rendering of the skin surface, noise due to motion artifacts, and an unknown spike artifact (horizontal bars above the ears). The "horns" are directly above the very high signal (and unavoidably moving) eyeballs. The noise at top dead center is probably due to CSF motion, strongest around the spinal cord axis. 
Middle: Rendering at a higher threshold, above most of the noise values. The signal strength falls off sharply toward the bottom planes, resulting in skin having less luminance than the noise, particularly at the ears and back of the head.
Bottom: Rendering after masking of noise, at the same rendering thresholds as the renderings at top.
Stereo pairs, cross-view.

MIP of superior noise and inferior signal of T2 scan
Comparison of noise at top of volume (most superior slices) with bottom and middle of volume.

Red (left) is a maximum intensity projection (MIP) of noise above the head. Bottom and middle slices (top, middle and right), and MIP (bottom) of inferior and inferior half slices, with noise map overlaid.

The noise is primarily from high signal regions in the inferior and central part of volume. A few particular structures, including the eyes, contribute most to the noise. Much of the noise is due to motion or flow artifacts, but some may be wrap around (spatial aliasing) from signal outside of the field of view.
Surface rendering parameters dialog boxes, for this example
Surface rendering parameters and dialog boxes, for this example.

    There are several ways of dealing these types of noise:

Use a high contrast/brightness threshold that suppresses the noise.
Often this threshold is not ideal for rendering purposes.
Any single threshold might be fine for some regions but not for others.
Manually erase noise by cropping and pen tools.
Tedious, hard to erase noise completely and uniformly, particularly near surfaces of interest.
Smooth the volume, which suppresses some scattered noise sources.
Fine surface features are also smoothed, and noise is not fully removed.
Often works well for quick and dirty renderings, hides irregularities.

    In this example masks are constructed and combined to remove noise outside the skin surface. It depends a skin surface value profile that is relatively uniform with high sub-surface values.

Select a threshold that suppresses all noise external to the skin surface, but also suppresses the surface of the skin (and some interior surfaces).

This threshold also suppresses low signal regions in the lower back of the head. A later step restores this portion.

mid sagital slice thresholded
A mid-sagital slice with a value threshold that removes most of the superior noise, but also removes inferior-posterior skin surface.
mid sagital slice thresholded
The same slice with a value threshold that retains inferior-posterior skin surface, but also retains a lot of superior noise contiguous with the skin.

Apply threshold transform to values, and save as a mask.

Remove spots of noise that are super-threshold or make a new mask by filling only contiguous regions of the mask.
Dilate the mask such that it overlaps the skin surface.

Open original volume as an overlay, move it to bottom of overlay stack.
Increase brightness/contrast of original volume so the outer skin surface is clearly visible.
Dilate the mask by rethresholding a gaussian smoothed copy.
Volume | Make Filtered Overlay | Gaussian FWHM = 3 voxels = 1.5 mm
Close original volume.
Threshold, check and adjust the smoothed mask.
Apply the threshold to the mask and save.

At this point the mask's effectiveness can be checked by applying the mask to the volume and rendering.
To Do: Show example and point out where the mask is not suitable.
Mask might need more or less dilation.
Some regions might need a different threshold.
Make separate masks and merge these masks (see below).

Make separate masks for regions that require different thresholds.
Restore the low signal region at the bottom of the volume.
Restore inferior-anterior surface (near upper lip)
Restore thin skin lateral to the right eye.

Merge separate masks.
Remember that with Merge All, what you see is what you get -- it operates on the transformed values, of all volumes, even though the transforms are not applied.
Adjust and feather mask boundary.
Threshold, check and adjust the smoothed mask.
Apply as default.
Save mask (before transform applied, so it can be modified)
Apply transform to mask
(Is this necessary for mask and apply mask?)
Colors | Apply Color Transforms to Values

Mask and apply mask to the original volume.

Overlays | Mask with this Overlay
Overlays | Merge All
Note: It is not necessary to apply transforms with Colors | Apply Color Transform to Values of the mask when merging. Overlays | Merge All operates on the transformed values, even though the transforms are not applied. 
Remember to remove transforms applied to base volume before using Merge All. Generally you don't want to alter the base volume values by any unapplied transforms.
Save masked volume.
File | Save As
000001_5_4_t2_tse3d_8b_iso_mskd3_G3f.vol.gz (24 MB)

See skin surface rendering parameters.

Skin surface renderings, after masking of noise:
skin surface rendering from T2-weighted MRI
Stereo pair, cross-view.
To Do: Details of
forehead artery
bumps on temple
sequence banding
eyelid thinness
lateral eye bone
tip of nose noise
ear, earphone depression
voxel aliasing

Video of full head rotation, 965 x 497 px,, 720 frames:
000001_5_4_t2_tse3d_8b_iso_mskd3_G3f_rot_720.vol.gz (Space native file format, 155 MB)
000001_5_4_t2_tse3d_8b_iso_mskd3_G3f_rot_720.avi (AVI video format, 61 MB)
000001_5_4_t2_tse3d_8b_iso_mskd3_G3f_rot_720.swf (Flash video format,  65 MB, runs in a browser window but has serious compression artifacts)

Stereo pair, cross-view.

Segmenting eye structures

axial slice of T2 MRI through eyes and optic nerve axial slice drawing through eyes, link to
Axial slice drawing through eyes from An Atlas of Anatomy,
by Florence Fenwick Miller, 1879. More info.
Axial slice of T2 MRI through eyes and optic nerves.

Eyeball interior (aqueous and vitreous humor)

To Do: The aqueous and vitreous humor ...
axial slice of T2 MRI through eyes and optic nerve

eye humor rendered on slice

eye humor and lenses rendered on slice
Schematic diagram of the human eye, link to
Top: Axial slice of T2 MRI through eyes and optic nerves.
Bottom: Rendered aqueous and vitreous humor surface rendering overlaid on slice.
from Schematic diagram of the human eye (Wikipedia)

Apply ROI mask

Make sure the mask is binary if desired. If the gaussian blur was used, this can be used to taper the edges in luminance.
Linear All? Maximum gamma and sweep through for low slice values? Color | Color index values should have a slider?
Tip: Don't mistakenly have the background very low values instead of black (0), because transparency of the masked images is altered. A zero value is always fully transparent, but any non-zero value is fully opaque by default.

If segmenting dark features on a light background, fill background with white.
For example, dark vessels on white background, with a white mask.
Overlays | Mask with this Overlay
Overlays | Merge All
Remember to remove transforms applied to base volume before using Merge All. With Merge All, what you see is what you get -- it operates on the transformed values, both of a mask and of masked volume(s), even though the transforms are not applied. 
File | Save As with "_mskd" appended to filestem.

Fix up masked ROI

To Do: Color and set defaults.
If the mask is subsequently altered, the
To Do: Handling extrema crossings in masked regions.
To Do: Surface estimates and gamma correction.
Post rendering brightness/contrast/gamma adjustment.
To Do: Resample to taste.

Render ROI

Use Colors | Brightness/Contrast/Gamma tools to threshold such that features are near saturated, and edges of masked features are near zero.
To Do: Volume | Make Filtered Overlay | Gaussian 1-2 voxels
To Do: Example rendering and lighting parameters.

[To Do: Multiple ROI composites, rendering, animation clips, and composites of animation clips.]

artery segmentation sequence slice

segmentation slice context
artery segmentation sequence:
slice of volume bounding a region of interest (ROI)
mask (cyan), partly transparent to assist manual editing
final binary mask (white)
masked volume slice with inverted luminance (red)
voxels interior to approximate rendered surface
The actual rendered surface is determined by the first gradient maximum along the line of sight.
Rendering of left artery segmentation auditory nerves Rendering of left artery segmentation behind raw volume data
Test render of segmented arteries near the brainstem/pons, from an anterior-superior perspective.
Resampled to .125 mm/vx. Stereo pair, cross-view.
Test render of segmented arteries near the brainstem, overlaid by raw thresholded render. From an anterior-inferior perspective, with cochlea is at upper right. Stereo pair, cross-view.
To Do: Rendering of colored volume behind partly transparent volume is incorrect (color info lost).
To Do: Rendering of colored volume behind hidden volume is incorrect (ghosting around edges).
To Do: Smoothed render at same, or all three orthogonal, orientations.
facial/acoustic nerve from superior perspective
facial/acoustic nerve from sagital perspective
internal acoustic meatus
vestibulocochlear nerve

Brainstem segmentation

[To Do: Primary techniques used.]

Rendering of orthogonal views of brainstem, link to, stereo pairs To Do: Show context of brainstem within head/brain. labelled cerrebelum illustration
Brainstem test segmentation, after first application of morphological and gaussian smoothing to the mask. Many minor flaws haven't been fixed up.
In reading order: from top, bottom, front, back, right and left.
(stereo pairs, cross-view)
"Inferior and Superior Surface of Cerrebellum", from "Surgical anatomy of the head and neck", by John Blair. Beautiful labelled illustration of a flattened cerebellum.

Data files

Raw DICOM files
scout (localizer) planes
"t2_tse3d_vfl_trans", the T2-weighted whole brain volume used in the Inner ear MRI example
"Cpd+t2_tse_tra_512_5mm", two T2-weighted whole brain volumes (with two complementary 3D contrasts using one sequence)
DTI (diffusion tensor imaging) data set

To Do: Zip and post these.
FSL NifTi 16-bit volume

Space volumes (Space Software native file format)
8-bit, adjusted

cropped regions of interest
8_bit regions
mask, gaussian smoothed masks
masked region of interest

Programs and code

Space Software

    Space is a free MS Windows (98, 2K, XP [Vista, W7 not tested but should work]) application for the display, navigation, rendering, editing, and measurement of 3-D data sets. In many respects it serves the same needs as a traditional 2-D image editor for 3-D data, but is organized and optimized for the particular requirements of handling 3-D data for research purposes. The program provides fast display of arbitrarily oriented planes through 3-D data, direct spatial correlation and combining of multiple spatially coincident data sets, several image processing operations, and an array of fully interactive display and rendering options for visualization.

MRIConvert (accessed 2/8/10)

    MRIConvert is a medical image file conversion utility that converts DICOM files to NIfTI 1.1, Analyze 7.5, SPM99/Analyze, BrainVoyager, and MetaImage volume formats. More information on how MRIConvert interprets these formats may be found  here. MRIConvert was written using wxWindows, an open-source library for cross-platform GUI development.

SS_Edges (Currently these 3-D segmentation examples don't utilize SS_edges directly, but some aspects of Space Software are based on SS_Edges methods.)

    Scale Space Edges is a freely available set of  MATLAB programs (m-files) that identifies maximal spatial intensity transitions in images (it's an edge detector).  Its design emphasizes edge segment continuity and accurate characterization of edges for use in image segmentation and feature extraction.  It can be configured to find the most salient edge segments early in the full search, and/or exhaustively search a given range of scale space.

    Edge points are defined as having a first derivative magnitude (2-D space) that is locally maximal (in scale space) in the first derivative direction.  The algorithm locates and follows edge segments, across the image and across scales, using a large set of oriented Difference of Gaussian filters to identify gradient directions and gradient maxima.


tonic vergence, dark vergence (bottom of page 4)

visual and optic axis:

Anatomy, extraocular muscles:

Is an Animal's Agility Affected by the Position of Its Eyes? (ScienceDaily, Feb. 23, 2010)

[T]he vestibulo-ocular reflex  is affected by the directions sensed by the canals and the pull directions of the eye muscles. In mammals, the eyes can be on the side of the head, as with rabbits, or at the front of the head like in cats, however the position of the canals is basically the same. In some mammals the brain must do extra calculations to adjust the signal from the canals to match the different pull directions of the eye muscles.


The team used MRI scanners to analyze the arrangement of canals and eye muscles in 51 species of mammal including giraffes, camels and zebra, tree shrews, bats and sloths. Astonishingly, the team found that the position of canals and eye muscles had no effect on the ability to see clearly at speed. In theory, a Sloth could travel as fast as a Cheetah without blurring its vision.


Comments are welcome (dow[at]