Practice and procedures for isolation and rendering
of structures and regions in volume (3-D) data using Space Software.
Currently a single example of isolating several
regions from a particular volumetric data set, from a T2-weighted
MRI sequence, is roughly
[To Do: Reference to edge detection material, animation clips, virion
segmentation, hippocampi ...]
To Do: Why Space Software? What
other tools are available, and what are their strengths and weaknesses?
Region of interest
definition based on complex volume data is difficult, requiring a
priori knowledge of structure and visual-manual intervention.This is an
processing problem called segmentation
. A variety of techniques
developed for 2-D images find natural extensions in 3-D.
Segmentation estimates of location
of surfaces, boundaries in a 3-D space, that usually correspond to
chemical-density interfaces in the material that was volumetrically
sampled. Potential locations of surface points can be automatically
generated with gradient operators -- surfaces are near inflection
points of gradient magnitude distribution. While Space Software
generate and use this type of gradient information, it is not
explicitly used in these examples. The rendering of surfaces does
implicitly use gradient information to represent surface.
3-D manual or
semi-automatic techniques require a suitable user
interface that Space Software provides. How is
Space Software used to define and
visualize regions of interest? How
could the software be improved and what auxilliary tools would be most
Here we are
concerned with selecting all or most volumetric elements that contain
surface points of a structure, and excluding surface points that are
not part of the structure. The method is to create and apply a mask to
the volume data, including only the desired surface points within a
A mask is layered on, or on a portion
of, the raw data volume. Semi-automatic and manual editing tools are
used to populate the mask. The mask is applied, and the remaining
surface elements are rendered, or represented and quantified.
Inner ear MRI example
This segmentation of
inner ear structures and nearby nerves and arteries from an MRI
T2-weighted image was done
primarily for the purpose of rendering, not volumetric assessment or
MRI volume data information
Subject: Mark Dow, T2 weighted brain
Date of scan: January 5, 2010
Siemens Allegra (3T head only), LCNI
, University of Oregon
phased array head coil (not the standard quadrature head coil)
Investigator: Mark Dow
for data set selection and conversion
single file (.nii) format
has supported input and
output of NIfTI-1 files since the release of version 3.3 in April 2006.
This is the second release of FSL that supports NIfTI-1, and more
features are implemented that in the previous release, although not
every feature is currently fully implemented.
In order to maintain backward
compatibility the current
version of FSL (3.3) also supports Analyze input and output.
As NIfTI-1 allows voxel coordinates to be mapped onto real-world (mm)
coordinates with a transformation (qform or sform) that either
preserves left-right orientation of the coordinate system (positive
or reverses left-right orientation (negative determinant) it was
necessary to assume that all Analyze files and any NIfTI
file which did not specify either
qform or sform,
had a reversed left-right orientation (also called "radiological
convention" in the FSL help pages). This was done in order to be
consistent with the avg152T1 standard image."
Create an adjusted
Space volume (.vol) format, 8-bit
To Do: Repost, and link to,
To Do: Fix the Space editing of
Volume | Volume Information |
Description and Title
comment text. Apparently the focus is
reinitialized, sending the cursor back to the corner at every key
Add volume info comment.
Volume | Volume
The point of this step is to maintain
important information about surfaces and textures in the conversion to
an 8-bit format.
It is handy to see a histogram of voxel luminance (values), native and
transformed, while making these adjustments.
Open the Colors | Transform and Histogram
Note that adjusting gamma affects the
as a function of luminance, an most surfaces are defined with respect
to relative gradient magnitudes.
For example, a positive gamma applied
to the volume will increase the number and strength of surface features
at low luminances compared with those at high luminance. Another way of
saying this is "contrast is increased for low luminance regions, and
decreased at high luminance regions".
[To Do: It would be nice to know a
gradient magnitude spectrum as a function of:
At inflection points only? ]
Select the full range of values that
are useful for segmentation.
| Apply Transform to Values
Compress to 8-bit values
Colors | Color Depth | 8-bit Grayscale
It is easiest to work with 8-bit values
that are numbered as 8-bit integers. Any scaling is equivalent, but
| Transform and Histogram | min. = 0 and max. = 255
Save, and save a copy
Most of the operations (creation of
masks and masked volumes) use this 8-bit volume. When performing merge
operations, it is easy to make the mistake of overwriting this file
with a modified version. The copy is used to easily recover from this
Append "_8-bit" to filestem, save in
Append "_copy" to filestem, save in
Choose one or more regions of interest
No good hints here, application
dependent. Don't bite off more than you can chew, but theres a cost to
endlessly nibbling tiny pieces.
It is better to segment largest
(superset) features first, as it can save some work at shared
But only if segmentation of the largest
features are useful.
Splicing multiple pieces together can
be done. View adjacent masks to insure continuity between volumes
bounding parts of the ROI (see below).
and dirty rendering of a region of interest (ROI)
It is handy to get an
idea of the
structure and data quality of a region and structures that are to be
segmented. It is difficult to get a feel for 3-D structure from slices
at arbitrary angles, and rendering a region in various ways helps
getting a hold on of the lay of the land. This can be done on the raw
16-bit volume, but it's a bit more efficient and just as good to use an
adjusted 8-bit volume.
Load volume into Space Software.
Load either an .nii or .vol format file
with File | Open Volume
Space does not read DICOM (.dcm) files.
Center volume origin on screen, if not
already centered, with Volume |
Volume Info | Center
This operation changes the Volume | Volume Info | Origin. Don't
resave this volume, or save as a unique copy, if you want to preserve
the original origin in the
Slice navigation -- looking at slices of
the volume data at any slice angle and position
left click drag for rotations
right click drag for translation in depth
To Do: ...
mouse (Colors |
Brightness/Contrast), or Colors
| Transform and Histogram
Crop and resample if
can make navigation while rendering easier if there is a lot of
distracting surroundings. Resampling to a smaller voxel size
(oversampling) allows for
less aliasing in the rendering, and can correct for anisotropic voxel
Select crop rectangle
To Do: ...
To Do: ...
Repeat cropping in third dimension
To Do: ...
Window the region of interest (if not
w-key (View | Window) and select rectangle
and/or Rendering | Rendering
parameters dialog box controls
Example quick and dirty
These examples were
cropping, and only oversampling to achieve an isotropic voxel size.
Less obstructed and chunky (aliased) renderings can be achieved
selective cropping and oversampling (To Do: Example.):
files, [To Do...]
| Resize (resample) | Isotropic x = .5, y = .5, z = .5 mm
(from raw voxel size of x = .5, y = .5, z = 1.0 mm
To Do: Add volume cues unique
to the windowed state to Space.
region of interest (vbROI)
Crop to vbROI
| Select Crop Rectangle
Volume | Crop
Do not change the origin. It will be
off center, but will stay registered with other ROIs.
Use windowing (View | Window
to center the image on the screen.
Volume Cues are not available in the windowed state,
To Do: Cropping does not work properly in the windowed state;
the crop rectangle is displayed at the wrong location.
| Resize (resample)
Set brightness and contrast defaults
b-key mouse, then Colors | Transform and Histogram | Apply as
These get changed frequently during
mask creation, so it's nice to easily return to a default view (Colors | Transform and Histogram | Default
All, or Colors | Use
Default Color Transform.
Rename and Save
All the manual
nitty-gritty work is
done here. Save often. Invariably this mask will require tweaking.
Create a blank overlay to hold the mask
) | New Blank Overlay
Overlap the current volume, 8-bit
| Transform and Histogram
to about 70%
Coloring (so mask voxels are easily distinguished):
channels White Point
to 1255 (just prepend
This has the effect of partially
red leaving the mask voxels cyan, so they can be distinguished easily
from underlying overlays.
| Save As
This is often tweaked later, and its
not convenient to reset these values every time.
Append "[name]_mask[#]" to filestem
in the selection mask
Edit | Edit Toolbar tools:
Select Edit Color
black or white
Gaussian blur for expansion
| Make Filtered Overlay | Gaussian
First rough cut
To Do: For regions with extensive
continuous contrast boundaries:
e.g. some inner ear structures
To Do: For large regions poor contrast boundaries:
Work with a copy of the vbROI
with a shortened filestem. Easier to recover from incorrect saving
When editing plane-by-plane:
Use "<" and ">" keys to navigate
to the next View | Slice Positioning
| Next or Previous Slice
Use an orthogonal orientation, o-key or View | Coronal, Sagital, or Axial
A lot of switching between mask and vbROI
The mask volume is being edited, but the contrast/brightness of the
vbROI is frequently changed.
| Overlay Viewer for selection of current volume, or get used to
n-key for cycling between
Also use the h-key (or
the "eye" button on the Overlay viewer) to toggle visibility of the
mask (Overlays | Hide this Overlay).
The Colors | Transform and Histogram
or Edit | Select Edit color
dialog boxes serve as good visual indicators of the current volume.
These dialog boxes are also used frequently for other tasks, e.g.
toggling back to default viewing parameters.
First cut 3-D fill
The fill tools operate on the displayed
luminance of the displayed over/underlays.
Set the brightness/contrast (b-key mouse position
) of the volume bounding this ROI
below the mask volume in the overlays list (Edit | Overlay Viewer
Remove excessive bleeding by redoing
after resetting the Max.
extent in the Edit | Tool properties
(with Fill 3-D
active) to a
If this isn't noticed before a lot of
editing is done, it is difficult to recover gracefully.
Try using Overlays | Mask with this Overlay
while editing the mask edges.
To Do: Rename, document, post.
Apply 3-D erosion and dilation operators to the mask
to remove unsupported voxels and fill in small gaps. This is currently
implemented as a MATLAB program, not as a part of Space Software. This
will reduce the need for manual
and make the regions that require manual intervention more obvious.
Save the mask in Space (.vol) format,
without g-zipped compression (.gz).
"_clean" volume in space
reset origin to match original mask
Save as the most current mask
If smoothing is used, return to binary
mask -- applying a feathered mask might work for bright to dark
boundaries, but has ill effects for dark to bright boundaries,
Use 5x5 circular Pen tool on one
feature or portion of a long edge, while stepping through planes.
To Do: Get a cheap pen tablet and make
it work with Space editing tools.
To Do: How to handle saddle points?
To Do: Isolating ROIs with a planar boundary.
ROI mask examples and particulars
Segmenting head from
This T2-weighted MRI volume
has noise and artifacts that are peculiar to the sequence, but not at
all uncommon for MRI in general. Often it is preferable to minimize the
effects of noise on rendering or surface determination.
volume renderings at with and without various noise artifacts:
|Top: Rendering of the skin
noise due to motion
artifacts, and an unknown spike artifact (horizontal bars above the
ears). The "horns" are directly above the very high signal (and
unavoidably moving) eyeballs. The noise at top dead center is probably
due to CSF motion, strongest around the spinal cord axis.
Middle: Rendering at a higher threshold, above most of the noise
signal strength falls off sharply toward the bottom planes, resulting
in skin having less luminance than the noise, particularly at the ears
and back of the head.
Bottom: Rendering after masking of noise, at the same rendering
thresholds as the renderings at top.
Stereo pairs, cross-view.
Comparison of noise at top of volume (most superior slices)
with bottom and middle of volume.
Red (left) is a maximum
intensity projection (MIP) of noise
above the head. Bottom and middle slices (top, middle and right), and
of inferior and inferior half slices, with noise map overlaid.
The noise is primarily from high signal regions in the inferior and
central part of volume. A few particular structures, including the
eyes, contribute most to the noise. Much of the noise is due to motion
or flow artifacts, but some may be wrap around (spatial aliasing) from
signal outside of the field of view.
Surface rendering parameters and dialog boxes, for this example.
There are several ways of dealing these types of
Use a high contrast/brightness threshold
that suppresses the noise.
Often this threshold is not ideal
for rendering purposes.
Any single threshold might be fine for
some regions but not for others.
Manually erase noise by cropping and
Tedious, hard to erase noise completely
and uniformly, particularly near surfaces of interest.
Smooth the volume, which suppresses
some scattered noise sources.
Fine surface features are also
smoothed, and noise is not fully removed.
Often works well for quick and dirty renderings, hides irregularities.
example masks are constructed and combined to remove noise outside
the skin surface. It depends a skin surface value profile that is
uniform with high sub-surface values.
Select a threshold that suppresses all
noise external to the skin surface, but also suppresses the surface of
the skin (and some interior surfaces).
This threshold also suppresses low signal regions in the lower back of
the head. A later step restores this portion.
A mid-sagital slice with a value threshold that removes most of the
superior noise, but also removes inferior-posterior skin surface.
The same slice with a value threshold that retains inferior-posterior
skin surface, but also retains a lot of superior noise contiguous with
Apply threshold transform to values,
and save as a mask.
Remove spots of noise that are super-threshold or make a new mask by
filling only contiguous regions of the mask.
Dilate the mask such that it overlaps
Open original volume as an overlay,
move it to bottom of overlay stack.
Increase brightness/contrast of original volume so the outer skin
surface is clearly visible.
Dilate the mask by rethresholding a gaussian smoothed copy.
| Make Filtered Overlay
| Gaussian FWHM = 3 voxels =
Close original volume.
Threshold, check and adjust the
Apply the threshold to the mask and save.
At this point the mask's effectiveness
can be checked by applying the mask to the volume and rendering.
Do: Show example and point out where the mask is not suitable.
Mask might need more or less dilation.
Some regions might need a different threshold.
Make separate masks and merge these
masks (see below).
Make separate masks for regions that require different thresholds.
Restore the low signal region at the
bottom of the volume.
Restore inferior-anterior surface (near upper lip)
Restore thin skin lateral to the right eye.
Merge separate masks.
that with Merge All
, what you see is what you
get -- it operates on the transformed values, of all volumes, even
though the transforms are not applied.
Adjust and feather mask boundary.
Threshold, check and adjust the
Apply as default.
Save mask (before transform applied, so
it can be modified)
Apply transform to mask
(Is this necessary for mask and apply
| Apply Color Transforms to Values
Mask and apply mask to the original volume
| Mask with this Overlay
Overlays | Merge All
Note: It is not
necessary to apply
transforms with Colors | Apply Color
Transform to Values
of the mask when merging. Overlays
| Merge All
operates on the transformed values, even though the
transforms are not applied.
Remember to remove
transforms applied to base volume before using Merge All. Generally you don't want
to alter the base volume values by any unapplied transforms.
Save masked volume.
surface renderings, after masking of noise:
Stereo pair, cross-view.
To Do: Details of
bumps on temple
lateral eye bone
tip of nose noise
ear, earphone depression
|Video of full head rotation, 965 x 497 px,, 720 frames:
(Space native file format, 155 MB)
(AVI video format, 61 MB)
(Flash video format, 65 MB,
runs in a browser window but has serious compression artifacts)
Stereo pair, cross-view.
To Do: The aqueous and vitreous humor
Apply ROI mask
Make sure the mask is binary if
desired. If the gaussian blur was used, this can be used to taper the
edges in luminance.
Linear All? Maximum gamma and sweep
through for low slice values? Color | Color index values should have a
Tip: Don't mistakenly have the
background very low values instead of black (0), because transparency
of the masked images is altered. A zero value is always fully
transparent, but any non-zero value is fully opaque by default.
If segmenting dark features on a light
background, fill background with white.
For example, dark vessels on white
background, with a white mask.
| Mask with this Overlay
Overlays | Merge All
Remember to remove
transforms applied to base volume before using Merge All. With Merge All, what you see is what you
get -- it operates on the transformed values, both of a mask and of
masked volume(s), even though the transforms are not applied.
File | Save As
appended to filestem.
Fix up masked ROI
To Do: Color and set defaults.
If the mask is subsequently altered, the
To Do: Handling extrema crossings in
To Do: Surface estimates and gamma correction.
To Do: Resample to taste.
| Brightness/Contrast/Gamma tools to threshold such that
features are near saturated, and edges of masked features are near zero.
To Do: Volume | Make Filtered Overlay
| Gaussian 1-2 voxels
To Do: Example rendering and lighting parameters.
[To Do: Multiple ROI composites, rendering, animation clips, and
composites of animation clips.]
[To Do: Primary techniques used.]
||To Do: Show context of brainstem
|Brainstem test segmentation,
after first application of morphological
and gaussian smoothing to the mask.
Many minor flaws haven't been fixed up.
In reading order: from top, bottom, front, back, right and left.
(stereo pairs, cross-view)
||"Inferior and Superior Surface of Cerrebellum", from "Surgical
anatomy of the head and neck", by John
Blair. Beautiful labelled illustration of a flattened
To Do: Zip and post these.
Raw DICOM files
scout (localizer) planes
T2-weighted whole brain volume used in the Inner ear MRI example
"Cpd+t2_tse_tra_512_5mm", two T2-weighted whole brain volumes (with two
complementary 3D contrasts using one sequence)
) data set
FSL NifTi 16-bit volume
Space volumes (Space Software
cropped regions of interest
mask, gaussian smoothed masks
masked region of interest
Programs and code
a free MS
2K, XP [Vista, W7 not tested but should work]) application for the
and measurement of 3-D data sets. In many respects it serves the same
needs as a traditional 2-D image editor for 3-D data, but is organized
optimized for the particular requirements of handling 3-D data for
research purposes. The program provides fast display of arbitrarily
oriented planes through 3-D data, direct spatial correlation and
combining of multiple spatially coincident data sets, several image
processing operations, and an array of fully interactive display and
rendering options for visualization.
3-D segmentation examples don't utilize SS_edges directly, but some
aspects of Space Software are based on SS_Edges methods.)
Scale Space Edges is a freely available set
of MATLAB programs
(m-files) that identifies maximal spatial intensity transitions in
images (it's an edge detector). Its design emphasizes edge
segment continuity and
accurate characterization of edges for use in image segmentation and
extraction. It can be configured to find the most salient edge
early in the full search, and/or exhaustively search a given range of
Edge points are defined as having a first
derivative magnitude (2-D
space) that is locally maximal (in scale space) in the first derivative
direction. The algorithm locates and follows edge segments,
across the image and across scales, using a large set of oriented
Difference of Gaussian filters to
identify gradient directions and gradient maxima.
tonic vergence, dark vergence
(bottom of page 4)
visual and optic axis:
Anatomy, extraocular muscles:
an Animal's Agility Affected by the Position of Its Eyes? (ScienceDaily, Feb. 23, 2010)
[T]he vestibulo-ocular reflex is
affected by the directions sensed by
the canals and the pull directions of the eye muscles. In mammals, the
eyes can be on the side of the head, as with rabbits, or at the front
of the head like in cats, however the position of the canals is
basically the same. In some mammals the brain must do extra
calculations to adjust the signal from the canals to match the
different pull directions of the eye muscles.
The team used MRI scanners to analyze the
arrangement of canals and eye
muscles in 51 species of mammal including giraffes, camels and zebra,
tree shrews, bats and sloths. Astonishingly, the team found that the
position of canals and eye muscles had no effect on the ability to see
clearly at speed. In theory, a Sloth could travel as fast as a Cheetah
without blurring its vision.
Comments are welcome (dow[at]uoregon.edu).