Mark
Dow
Space
Software
Volume
segmentation notes
Practice and procedures for isolation and rendering
of structures and regions in volume (3-D) data using Space Software.
Currently a single example of isolating several
regions from a particular volumetric data set, from a T2-weighted
MRI sequence, is roughly
documented.
Programs
and code
[To Do: Reference to edge detection material, animation clips, virion
segmentation, hippocampi ...]
Background
Region of interest
definition based on complex volume data is difficult, requiring a
priori knowledge of structure and visual-manual intervention.This is an
image
processing problem called segmentation. A variety of techniques
developed for 2-D images find natural extensions in 3-D.
Segmentation estimates of location
of surfaces, boundaries in a 3-D space, that usually correspond to
chemical-density interfaces in the material that was volumetrically
sampled. Potential locations of surface points can be automatically
generated with gradient operators -- surfaces are near inflection
points of gradient magnitude distribution. While
Space Software
does
generate and use this type of gradient information, it is not
explicitly used in these examples. The rendering of surfaces does
implicitly use gradient information to represent surface.
3-D manual or
semi-automatic techniques require a suitable user
interface that Space Software provides. How is
Space Software used to define and
visualize regions of interest? How
could the software be improved and what auxilliary tools would be most
useful?
Basic ideas
Here we are
concerned with selecting all or most volumetric elements that contain
surface points of a structure, and excluding surface points that are
not part of the structure. The method is to create and apply a mask to
the volume data, including only the desired surface points within a
binary mask.
A mask is layered on, or on a portion
of, the raw data volume. Semi-automatic and manual editing tools are
used to populate the mask. The mask is applied, and the remaining
surface elements are rendered, or represented and quantified.
Inner ear MRI example
This segmentation of
inner ear structures and nearby nerves and arteries from an MRI
T2-weighted image was done
primarily for the purpose of rendering, not volumetric assessment or
other measurement.
Image acquisition
MRI volume data information
Subject: Mark Dow, T2 weighted brain
Date of scan: January 5, 2010
MRI system:
Siemens Allegra (3T head only),
LCNI, University of Oregon
phased array head coil (not the standard quadrature head coil)
Sequence: t2_tse3d_vfl_trans
Investigator: Mark Dow
MRIConvert
for data set selection and conversion
FSL NifTi
single file (.nii) format
"FSL
has supported input and
output of NIfTI-1 files since the release of version 3.3 in April 2006.
This is the second release of FSL that supports NIfTI-1, and more
features are implemented that in the previous release, although not
every feature is currently fully implemented.
In order to maintain backward
compatibility the current
version of FSL (3.3) also supports Analyze input and output.
As NIfTI-1 allows voxel coordinates to be mapped onto real-world (mm)
coordinates with a transformation (qform or sform) that either
preserves left-right orientation of the coordinate system (positive
determinant)
or reverses left-right orientation (negative determinant) it was
necessary to assume that all Analyze files and any NIfTI
file which did not specify either
qform or sform,
had a reversed left-right orientation (also called "radiological
convention" in the FSL help pages). This was done in order to be
consistent with the avg152T1 standard image."
Create an adjusted
Space volume (.vol) format, 8-bit
To Do: Repost, and link to,
file specification.
Add volume info comment.
Volume | Volume
Information
To Do: Fix the Space editing of
Volume | Volume Information |
Description and Title
comment text. Apparently the focus is
reinitialized, sending the cursor back to the corner at every key
stroke.
Adjust brightness/contrast/gamma
The point of this step is to maintain
important information about surfaces and textures in the conversion to
an 8-bit format.
It is handy to see a histogram of voxel luminance (values), native and
transformed, while making these adjustments.
Open the Colors | Transform and Histogram
dialog box.
Colors
| Gamma
Note that adjusting gamma affects the
gradient distribution
as a function of luminance, an most surfaces are defined with respect
to relative gradient magnitudes.
For example, a positive gamma applied
to the volume will increase the number and strength of surface features
at low luminances compared with those at high luminance. Another way of
saying this is "contrast is increased for low luminance regions, and
decreased at high luminance regions".
[To Do: It would be nice to know a
gradient magnitude spectrum as a function of:
scale
luminance
At inflection points only? ]
Colors
| Brightness/Contrast
Select the full range of values that
are useful for segmentation.
Apply transform
Colors
| Apply Transform to Values
Compress to 8-bit values
Colors | Color Depth | 8-bit Grayscale
Rescale (min/max),?
It is easiest to work with 8-bit values
that are numbered as 8-bit integers. Any scaling is equivalent, but
more confusing.
Colors
| Transform and Histogram | min. = 0 and max. = 255
Save, and save a copy
Most of the operations (creation of
masks and masked volumes) use this 8-bit volume. When performing merge
operations, it is easy to make the mistake of overwriting this file
with a modified version. The copy is used to easily recover from this
error.
File |
Save As
Append "_8-bit" to filestem, save in
.vol.gz format.
Append "_copy" to filestem, save in
.vol.gz format.
Choose one or more regions of interest
for segmentation.
No good hints here, application
dependent. Don't bite off more than you can chew, but theres a cost to
endlessly nibbling tiny pieces.
It is better to segment largest
(superset) features first, as it can save some work at shared
boundaries.
But only if segmentation of the largest
features are useful.
Splicing multiple pieces together can
be done. View adjacent masks to insure continuity between volumes
bounding parts of the ROI (see below).
Quick
and dirty rendering of a region of interest (ROI)
It is handy to get an
idea of the
structure and data quality of a region and structures that are to be
segmented. It is difficult to get a feel for 3-D structure from slices
at arbitrary angles, and rendering a region in various ways helps
getting a hold on of the lay of the land. This can be done on the raw
16-bit volume, but it's a bit more efficient and just as good to use an
adjusted 8-bit volume.
Load volume into Space Software.
Load either an .nii or .vol format file
with File | Open Volume
Space does not read DICOM (.dcm) files.
Center volume origin on screen, if not
already centered, with Volume |
Volume Info | Center
This operation changes the Volume | Volume Info | Origin. Don't
resave this volume, or save as a unique copy, if you want to preserve
the original origin in the
header information.
Slice navigation -- looking at slices of
the volume data at any slice angle and position
left click drag for rotations
right click drag for translation in depth
To Do: ...
Set brightness/contrast
b-key
mouse (Colors |
Brightness/Contrast), or Colors
| Transform and Histogram
Crop and resample if
desired. Cropping
can make navigation while rendering easier if there is a lot of
distracting surroundings. Resampling to a smaller voxel size
(oversampling) allows for
less aliasing in the rendering, and can correct for anisotropic voxel
size.
Select crop rectangle
To Do: ...
Crop
To Do: ...
Repeat cropping in third dimension
To Do: ...
Window the region of interest (if not
cropped appropriately)
w-key (View | Window) and select rectangle
Render
r-key,
and/or Rendering | Rendering
parameters dialog box controls
Example quick and dirty
renderings
These examples were
done without
cropping, and only oversampling to achieve an isotropic voxel size.
Less obstructed and chunky (aliased) renderings can be achieved
selective cropping and oversampling (To Do: Example.):
See data
files, [To Do...]
Volume
| Resize (resample) | Isotropic x = .5, y = .5, z = .5 mm
(from raw voxel size of x = .5, y = .5, z = 1.0 mm
Volume bounding
a
region of interest (vbROI)
Crop to vbROI
Volume
| Select Crop Rectangle
Volume | Crop
Do not change the origin. It will be
off center, but will stay registered with other ROIs.
Use windowing (
View | Window)
to center the image on the screen.
View |
Volume Cues are not available in the windowed state,
unfortunately.
To Do: Add volume cues unique
to the windowed state to Space.
To Do: Cropping does not work properly in the windowed state;
the crop rectangle is displayed at the wrong location.
Resize
Volume
| Resize (resample)
Set brightness and contrast defaults
b-key mouse, then Colors | Transform and Histogram | Apply as
Default.
These get changed frequently during
mask creation, so it's nice to easily return to a default view (Colors | Transform and Histogram | Default
All, or Colors | Use
Default Color Transform.
Rename and Save
All the manual
nitty-gritty work is
done here. Save often. Invariably this mask will require tweaking.
Create a blank overlay to hold the mask
Overlays
(or
File)
| New Blank Overlay
Overlap the current volume, 8-bit
grayscale
Colors
| Transform and Histogram
Transparency
to about 70%
Coloring (so mask voxels are easily distinguished):
Red channels
White Point to 1255 (just prepend
the 1).
This has the effect of partially
removing
red leaving the mask voxels cyan, so they can be distinguished easily
from underlying overlays.
Apply
as Default
This is often tweaked later, and its
not convenient to reset these values every time.
File
| Save As and rename
Append "[name]_mask[#]" to filestem
Fill
in the selection mask
Use Edit,
Edit | Edit Toolbar tools:
Edit |
Select Edit Color
black or white
Fill
3-D
Tool
properties
Fill
(2-D)
Tool
properties
Pen
Tool
properties
Gaussian blur for expansion
Volume
| Make Filtered Overlay | Gaussian
First rough cut
To Do: For regions with extensive
continuous contrast boundaries:
e.g. some inner ear structures
To Do: For large regions poor contrast boundaries:
e.g. brainstem
Tips:
Work with a copy of the
vbROI
with a shortened filestem. Easier to recover from incorrect saving
mistakes.
When editing plane-by-plane:
Use "<" and ">" keys to navigate
to the next View | Slice Positioning
| Next or Previous Slice
Use an orthogonal orientation, o-key or View | Coronal, Sagital, or Axial
A lot of switching between mask and
vbROI is required.
The mask volume is being edited, but the contrast/brightness of the
vbROI is frequently changed.
Use Edit
| Overlay Viewer for selection of current volume, or get used to
n-key for cycling between
volumes.
Also use the h-key (or
the "eye" button on the Overlay viewer) to toggle visibility of the
mask (Overlays | Hide this Overlay).
The Colors | Transform and Histogram
or Edit | Select Edit color
dialog boxes serve as good visual indicators of the current volume.
These dialog boxes are also used frequently for other tasks, e.g.
toggling back to default viewing parameters.
First cut 3-D fill
The fill tools operate on the displayed
luminance of the displayed over/underlays.
Set the brightness/contrast (
b-key mouse position) of the
volume bounding this ROI,
below the mask volume in the overlays list (
Edit | Overlay Viewer).
Remove excessive bleeding by redoing
after resetting the
Max. voxel
extent in the
Edit | Tool properties
(with
Fill 3-D active) to a
smaller value.
If this isn't noticed before a lot of
editing is done, it is difficult to recover gracefully.
Try using Overlays | Mask with this Overlay
while editing the mask edges.
Morphological
smoothing
Apply 3-D erosion and dilation operators to the mask
to remove unsupported voxels and fill in small gaps. This is currently
implemented as a MATLAB program, not as a part of Space Software. This
will reduce the need for
manual
fine tuning
and make the regions that require manual intervention more obvious.
Save the mask in Space (.vol) format,
without g-zipped compression (.gz).
Apply operators:
MATLAB command:
To Do: Rename, document, post.
Open
"_clean" volume in space
reset origin to match original mask
Save as the most current mask
Gaussian
smoothing
If smoothing is used, return to binary
mask -- applying a feathered mask might work for bright to dark
boundaries, but has ill effects for dark to bright boundaries,
Fine tuning:
Use 5x5 circular Pen tool on one
feature or portion of a long edge, while stepping through planes.
To Do: Get a cheap pen tablet and make
it work with Space editing tools.
To Do: How to handle saddle points?
To Do: Isolating ROIs with a planar boundary.
ROI mask examples and particulars
Segmenting head from
exterior noise
This
T2-weighted MRI volume
has noise and artifacts that are peculiar to the sequence, but not at
all uncommon for MRI in general. Often it is preferable to minimize the
effects of noise on rendering or surface determination.
Whole
volume renderings at with and without various noise artifacts:
 |
Top: Rendering of the skin
surface,
noise due to motion
artifacts, and an unknown spike artifact (horizontal bars above the
ears). The "horns" are directly above the very high signal (and
unavoidably moving) eyeballs. The noise at top dead center is probably
due to CSF motion, strongest around the spinal cord axis.
Middle: Rendering at a higher threshold, above most of the noise
values. The
signal strength falls off sharply toward the bottom planes, resulting
in skin having less luminance than the noise, particularly at the ears
and back of the head.
Bottom: Rendering after masking of noise, at the same rendering
thresholds as the renderings at top.
Stereo pairs, cross-view. |
|
Comparison of noise at top of volume (most superior slices)
with bottom and middle of volume.
Red (left) is a maximum
intensity projection (MIP) of noise
above the head. Bottom and middle slices (top, middle and right), and
MIP (bottom)
of inferior and inferior half slices, with noise map overlaid.
The noise is primarily from high signal regions in the inferior and
central part of volume. A few particular structures, including the
eyes, contribute most to the noise. Much of the noise is due to motion
or flow artifacts, but some may be wrap around (spatial aliasing) from
signal outside of the field of view. |
Surface rendering parameters and dialog boxes, for this example. |
There are several ways of dealing these types of
noise:
Use a
high contrast/brightness threshold
that suppresses the noise.
Often this threshold is not ideal
for rendering purposes.
Any single threshold might be fine for
some regions but not for others.
Manually erase noise by cropping and
pen tools.
Tedious, hard to erase noise completely
and uniformly, particularly near surfaces of interest.
Smooth the volume, which suppresses
some scattered noise sources.
Fine surface features are also
smoothed, and noise is not fully removed.
Often works well for quick and dirty renderings, hides irregularities.
In this
example masks are constructed and combined to remove noise outside
the skin surface. It depends a skin surface value profile that is
relatively
uniform with high sub-surface values.
Select a threshold that suppresses all
noise external to the skin surface, but also suppresses the surface of
the skin (and some interior surfaces).
This threshold also suppresses low signal regions in the lower back of
the head. A later step restores this portion.
A mid-sagital slice with a value threshold that removes most of the
superior noise, but also removes inferior-posterior skin surface. |
The same slice with a value threshold that retains inferior-posterior
skin surface, but also retains a lot of superior noise contiguous with
the skin.
|
Apply threshold transform to values,
and save as a mask.
Remove spots of noise that are super-threshold or make a new mask by
filling only contiguous regions of the mask.
Dilate the mask such that it overlaps
the
skin surface.
Open original volume as an overlay,
move it to bottom of overlay stack.
Increase brightness/contrast of original volume so the outer skin
surface is clearly visible.
Dilate the mask by rethresholding a gaussian smoothed copy.
Volume
| Make Filtered Overlay
| Gaussian FWHM = 3 voxels =
1.5 mm
Close original volume.
Threshold, check and adjust the
smoothed mask.
Apply the threshold to the mask and save.
At this point the mask's effectiveness
can be checked by applying the mask to the volume and rendering.
To
Do: Show example and point out where the mask is not suitable.
Mask might need more or less dilation.
Some regions might need a different threshold.
Make separate masks and merge these
masks (see below).
Make separate masks for regions that require different thresholds.
Restore the low signal region at the
bottom of the volume.
Restore inferior-anterior surface (near upper lip)
Restore thin skin lateral to the right eye.
Merge separate masks.
Remember that with
Merge All, what you see is what you
get -- it operates on the transformed values, of all volumes, even
though the transforms are not applied.
Adjust and feather mask boundary.
Threshold, check and adjust the
smoothed mask.
Apply as default.
Save mask (before transform applied, so
it can be modified)
Apply transform to mask
(Is this necessary for mask and apply
mask?)
Colors
| Apply Color Transforms to Values
Mask and apply mask to the original volume.
Overlays
| Mask with this Overlay
Overlays | Merge All
Note: It is
not necessary to apply
transforms with
Colors | Apply Color
Transform to Values of the mask when merging.
Overlays
| Merge All operates on the transformed values, even though the
transforms are not applied.
Remember to remove
transforms applied to base volume before using Merge All. Generally you don't want
to alter the base volume values by any unapplied transforms.
Save masked volume.
File |
Save As
Skin
surface renderings, after masking of noise:
Stereo pair, cross-view. |
To Do: Details of
forehead artery
bumps on temple
sequence banding
eyelid thinness
lateral eye bone
tip of nose noise
ear, earphone depression
voxel aliasing
|
|
|
Video of full head rotation, 965 x 497 px,, 720 frames:
000001_5_4_t2_tse3d_8b_iso_mskd3_G3f_rot_720.vol.gz
(Space native file format, 155 MB)
000001_5_4_t2_tse3d_8b_iso_mskd3_G3f_rot_720.avi
(AVI video format, 61 MB)
000001_5_4_t2_tse3d_8b_iso_mskd3_G3f_rot_720.swf
(Flash video format, 65 MB,
runs in a browser window but has serious compression artifacts)
Stereo pair, cross-view. |
|
Segmenting
eye structures
To Do: The aqueous and vitreous humor
...
Apply ROI mask
Make sure the mask is binary if
desired. If the gaussian blur was used, this can be used to taper the
edges in luminance.
Linear All? Maximum gamma and sweep
through for low slice values? Color | Color index values should have a
slider?
Tip: Don't mistakenly have the
background very low values instead of black (0), because transparency
of the masked images is altered. A zero value is always fully
transparent, but any non-zero value is fully opaque by default.
If segmenting dark features on a light
background, fill background with white.
For example, dark vessels on white
background, with a white mask.
Overlays
| Mask with this Overlay
Overlays | Merge All
Remember to remove
transforms applied to base volume before using Merge All. With Merge All, what you see is what you
get -- it operates on the transformed values, both of a mask and of
masked volume(s), even though the transforms are not applied.
File | Save As with "_mskd"
appended to filestem.
Fix up masked ROI
To Do: Color and set defaults.
If the mask is subsequently altered, the
To Do: Handling extrema crossings in
masked regions.
To Do: Surface estimates and gamma correction.
Post rendering
brightness/contrast/gamma adjustment.
To Do: Resample to taste.
Render ROI
Use Colors
| Brightness/Contrast/Gamma tools to threshold such that
features are near saturated, and edges of masked features are near zero.
To Do: Volume | Make Filtered Overlay
| Gaussian 1-2 voxels
To Do: Example rendering and lighting parameters.
[To Do: Multiple ROI composites, rendering, animation clips, and
composites of animation clips.]
Brainstem
segmentation
[To Do: Primary techniques used.]
 |
To Do: Show context of brainstem
within head/brain. |
 |
Brainstem test segmentation,
after first application of morphological
and gaussian smoothing to the mask.
Many minor flaws haven't been fixed up.
In reading order: from top, bottom, front, back, right and left.
(stereo pairs, cross-view) |
|
"Inferior and Superior Surface of Cerrebellum", from "Surgical
anatomy of the head and neck", by John
Blair. Beautiful labelled illustration of a flattened
cerebellum. |
Data files
Raw DICOM files
inlcudes:
scout (localizer) planes
"
t2_tse3d_vfl_trans", the
T2-weighted whole brain volume used in the
Inner ear MRI example
"Cpd+t2_tse_tra_512_5mm", two T2-weighted whole brain volumes (with two
complementary 3D contrasts using one sequence)
DTI (
diffusion
tensor imaging) data set
FSL NifTi 16-bit volume
Space volumes (
Space Software native
file format)
16-bit:
cropped regions of interest
8_bit regions
mask, gaussian smoothed masks
masked region of interest
Programs and code
Space
Software
Space is
a free MS
Windows (98,
2K, XP [Vista, W7 not tested but should work]) application for the
display, navigation,
rendering, editing,
and measurement of 3-D data sets. In many respects it serves the same
needs as a traditional 2-D image editor for 3-D data, but is organized
and
optimized for the particular requirements of handling 3-D data for
research purposes. The program provides fast display of arbitrarily
oriented planes through 3-D data, direct spatial correlation and
combining of multiple spatially coincident data sets, several image
processing operations, and an array of fully interactive display and
rendering options for visualization.
MRIConvert
(accessed 2/8/10)
SS_Edges (Currently these
3-D segmentation examples don't utilize SS_edges directly, but some
aspects of Space Software are based on SS_Edges methods.)
Scale Space Edges is a freely available set
of MATLAB programs
(m-files) that identifies maximal spatial intensity transitions in
images (it's an edge detector). Its design emphasizes edge
segment continuity and
accurate characterization of edges for use in image segmentation and
feature
extraction. It can be configured to find the most salient edge
segments
early in the full search, and/or exhaustively search a given range of
scale
space.
Edge points are defined as having a first
derivative magnitude (2-D
space) that is locally maximal (in scale space) in the first derivative
direction. The algorithm locates and follows edge segments,
across the image and across scales, using a large set of oriented
Difference of Gaussian filters to
identify gradient directions and gradient maxima.
References
tonic vergence, dark vergence
http://arapaho.nsuok.edu/~salmonto/vs3_materials/Lecture4.pdf
(bottom of page 4)
http://www3.interscience.wiley.com/journal/119302623/abstract
http://cat.inist.fr/?aModele=afficheN&cpsidt=2736263
http://www.biomedsearch.com/nih/Resting-positions-vergence-their-relation/17523043.html
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6T0W-3Y0S4DT-28&_user=10&_coverDate=01%2F31%2F1996&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1216372827&_rerunOrigin=google&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=ad9300b9ea204095e96345a69a791e09
visual and optic axis:
http://books.google.com/books?id=MHgx-jBA0TAC&pg=PA30&lpg=PA30&dq=optic+axis+human+eye&source=bl&ots=DGGxrPG893&sig=5cf4tCNGCRbH80BFR2-92A15ik0&hl=en&ei=t9WCS9OqGo7UsgOXspCUBA&sa=X&oi=book_result&ct=result&resnum=6&ved=0CBQQ6AEwBTgK#v=onepage&q=&f=false
http://www.telescope-optics.net/eye.htm
http://www.springerlink.com/content/n0527771p0278470/
http://en.wikipedia.org/wiki/Stiles%E2%80%93Crawford_effect
http://adsabs.harvard.edu/abs/1997SPIE.3208..302L
http://www.springerlink.com/content/p772p9133513m174/
http://webvision.med.utah.edu/anatomy.html
Is
an Animal's Agility Affected by the Position of Its Eyes? (ScienceDaily, Feb. 23, 2010)
[T]he vestibulo-ocular reflex is
affected by the directions sensed by
the canals and the pull directions of the eye muscles. In mammals, the
eyes can be on the side of the head, as with rabbits, or at the front
of the head like in cats, however the position of the canals is
basically the same. In some mammals the brain must do extra
calculations to adjust the signal from the canals to match the
different pull directions of the eye muscles.
...
The team used MRI scanners to analyze the
arrangement of canals and eye
muscles in 51 species of mammal including giraffes, camels and zebra,
tree shrews, bats and sloths. Astonishingly, the team found that the
position of canals and eye muscles had no effect on the ability to see
clearly at speed. In theory, a Sloth could travel as fast as a Cheetah
without blurring its vision.
...
Comments are welcome (dow[at]uoregon.edu).
